~o10J

Simplex Optimizatio

v. Zitko

Biological Station,

S1. Andrews, N. B., EOG 2XO

September 1984

Canadian Technical Report of

Fisheries and Aquatic Scienceso. 1308

FISheries Ptkhesand OCeans et Oceans

Canadian Technical Report ofFi herie and quatic cience

Te hni al report c nt in

rapp rt te

Rapport technique canadien decience halieutique et aquatique

.Lnt

pUl

Canadian Technical Report of

Fisheries and Aquatic Sciences 1308

September 19114

SIMPLEX OPTIMIZATION

by

V. Zitko

Fisheries and Environmental Sciences

Fisheries Research Branch

Department of Fisheries and Oceans

Biological Station

S to Andrews. N. B. EOG 2XO Canada

This is the one hundred and seventy-fifth Technical Report from

the Biological Station. St. Andrews. N. B.

11

©Minister of Supply ~nd Services Canada 1984

Correct citation for this publication:

Zitko, V. 1984. Simplex optimization. Can. Tech. Rep. Fish. Aquat. Sci. 1308: iii + 18 p.

iii

ABSTRACT

Zitko, V. 1984. Simplex optimization. Can. Tech. Rep. Fish. Aquat. Sci. l308: iii + Hi p.

The simplex optimization algorithm is reviewed and a program written in HP-BASIC is presented. Theprogram includes several recent modifications of the simplex algorithm and options for dat'! input, initialsimplex selection, summary report, and study of the optimized function.

RESUME

Zitko, V. 1984. Simplex optimiztion. Can. Tech. Rep. Fish. Aquat. Sci. 1308: iii + 18 p.

Ce rapport contient une revue de l'algorithme simplex d'optimisation ainsi qu'un programme machine ecriten HP-BASIC. Le programme inclut plusieurs modi ficat ions recentes apportees ai' algorithme simplex, de memeque diverses options pour l'entree des donnees. 11 comprend en outre une selection simplex initiale, unresume de rapport et une etude de la fonction optimisee.

INTRODUCTION

The objective of this report is to provide aversatile simplex program for general laboratoryapplications ranging from curve fitting to tuning ofchemical analytical instruments.

Simplex optimization is a direct search methodto find an extreme response. It is "direct" sinceit does not require the determination of slope(s)and "search" because it does not need a descriptionof the response surface. The extreme response ismost frequently a minimum response (for example,minimum deviation in fitting functions to data). Itmay be maximum response when dealing with the yieldof a chemical reaction, instrument signal or peakseparation in chromatography.

The examples above indicate the variety oflaboratory applications such as chromatographicseparation improvement by peak resolut ion (Cela andPerez-Bustamante 1983) or by actual separation opti­mization (Berridge 1984), reaction yield improvement(Dean et al. 1975), analytical methods development(Morgan and Deming 1974; Backman and Shanbag 1984),instrumental parameters optimization (Routh et al.1977; Leggett 1983; Burgess and Hayumbu 1984), drugdesign (Darvas et al. 1978), and pattern recognition(Ritter et al. 1975; Kaberline and Wilkins 1978).

1

Surprisingly, neither SPSS (Nie et al. 1975)nor IMSL (IMSL 1982) contains simplex-basedoptimization programs.

SIMPLEX ALGORITHM

Simplex optimization was presented toanalytical chemists by Deming and Morgan (1973) andto fisheries biologists by Schnute (1982). Aconcise description by Shavers et al. (1979) isavailable and almost all the papers quoted contain abrief summary of simplex search rules.

Simplex is an n-dimensional object with (n+l)vertices; a triangle in two, a tetrahedron inthree, dimensions. The vertices are sets of valuesof parameters (n parameters per vertex) at which theresponse is determined. The best (B), worst (W) andnext-to-worst (N) vertices are located. The ~,an ofall vertices except W is the centroid (C, Fig.1).Points NC, C, PC, R, and E are negative contrac­tion, centroid, positive contraction, reflection,and expansion, respectively. The mean of allvertices (not shown) is the center of gravity.

As can be seen from Fig. 1, all the points ofpotential simplex movement (NC, PC, R, and E ) lieon the straight line through Wand C, given by Eq.1 :

where Point=NC, PC, R, or E and F=-0.5, 0.5, 1 or 2,respectively.

Consider for example a two-dimensional simplexof the func t ion z = 2 - x**2 y**2:

Computer programs for simplex optimization arenot readily available. A FORTRAN listing based onthe original algorithm was published (O'Neill 1971).A modified version including I/O procedures wasdeveloped by Olsson (1974). The Tektronix statis­tics package (Tektronix 1975) contains a version ofthe original algorithm in Tektronix BASIC.

The newer simplex algorithms are available inan ALGOL-like notation from Ryan et a1. (1980) andcommercial programs are obtainable from van der Wielet al. (1983). Simplex algorithms are also includedas parts of pharmacokinetic programs (Yamaoka et al.1981; Zitko 1982).

NII

II 1I

(Eq. 1)

xy

z

11

o

Point

3i

-s

C + F*(C-W),

23

-11

1I

1I II E

II--------X--------X--------X--------X----------------XII II1I 1I

II II

II it

II II

II

B

Fig. 1. A two-dimensional simplex. Responses in verticesare: worst (W), next-to-worst (N), and best (B). NC, C, PC,R, and E indicate negative contraction, centroid, positivecontraction, reflection, and expansion, respectively. Forsimplex rules, see text and Fig. 2.

2

n ; number of vertices.

SU1PLEX MODIFICATIONS

Centroid need not be calculated since it can beexpressed in terms of center of gravity and theworst vertex (Eq. 3):

The best response (0) is at x=y=1; the worst(-11) is at x=2, y=3. The coordinates of Care x=2and y=1 [(1+3)/2;(1+1)/2] and those of Rare x=2 andy;-l [2+(2-2);1+(1-3)].

PROGRAH DESCRIPTION

lead to faster convergence by providing a moredirect path to the extreme sought. According tosome authors (see, for example, Aberg and Gustavsson1982) this modification does not outperform theconventional simplex, particularly with noisy data.

An example of the interactive input ispresented in Appendix 2. For stereotype routineapplications it is of advantage to change the INPUTstatements into REMarks and to establish the desiredvalues of the variables in the statement followingthe INPUT statement. Whenever possible, the programshould be operated in compiled BASIC.

The program SX (see listing in Appendix 1)incorporates the modifications expressed by Eq. 2-4.The simplex algorithm is called as a subroutine froma master program. The latter establishesoperational parameters (type of extreme, number ofsimplex evaluations, convergence criterium, andfrequency of printing intermediate results, etc.).The default values are minimum, 120, lE-4, and 20,respectively. Data input may be from file (forformat see below), manually, or, in the case offinding the minimum of a func tion, data are notneeded. The master program also selects the initialsimplexes (several, default 10) to ensure that thesimplex convergence is consistent. Initialsimplexes may be written to a file together with thefinal best value. This may be useful in studyingthe properties of the optimized function.

Appendix 2 also contains an example of theoutput for function number 4, described later inthis report. The output gives the number ofattempts needed to establish the initial simplexwi thin the specified range of variables and theactual initial simplex. Following this, the lettercodes record operations performed on the simplexes.E, R, PC, and NC are as defined, Prn and Nrnindicate positive (negative) contraction followed byreflection from the next-to-worst vertex. Interimresults (simplex and responses) are printed next,followed by the Hnal resul t ("converged" or "notconverged"). The program finishes by printing asummary table.

DATA INPUTCentroid;(n*[Center gravity]­

[Worst vertex])/(n-l)

[Center gravity]ne~[Center gravity]old +([Vertex]new-[Vertex]old)/n

The simplex search rules have changed somewhatover the years, mainly as a result of modificationssuggested by O'Neill (1971), Brookes et al. (1981),and Aberg and Gus tavsson (1982). The algorithmoutlined in Fig. 2 is that of Aberg and Gustavsson(1982), except for the "RSP.BT." test. This hasbeen added to stop the algorithm when there is nofurther improvement in response at the vertices.The addition of this test improved the performanceof the algorithm in optimizations described later inthis report.

Super Modified Simplex (SMS, Routh et al.1977) locates the worst vertex (W), centroid (C),and reflection (R). It determines the responses atC and R (that at W is available), and fits a seconddegree. polynomial to these three values. A newvertex is then located at the minimum (maximum) ofthis polynomial or close to a preselec ted boundaryalong the W-C-R line. If the response at the latteris not better than that at R, the response at R isretained. In many situations ~IS appears to befaster than the usual simplex algorithm.

Additional improvement in speed is achieved bymodifications in calculating the centroid (Brisseyet al. 1979). According to these authors, center ofgravity rather than the centroid is calculated forthe initial simplex. It is then updated as requiredby adding the fraction of the difference between thenew and the old vertex (Eq. 2):

(Eq. 3)

(Eq. 2)

located the lineand the vertex (Eq.

Standard data file for curve fitting has thebelow:

Another modification includes the weighting ofvertices during the calculation of the centroid(Keefer 1973; Ryan et al. 1980). This technique may

F ; 0.5, 1.5, 2, or 3 for negative contraction,positive contraction, reflection, or expansion,respectively, in the standard simplex, or isdetermined from the quadratic polynomial in SMS.Eq. 3 can be used instead of explicit calculation ofcentroid in Eq. 4.

<--- number of points<-- independent and dependent

variable(starting in columns 1and 15, respectively)

5

101932

4o

123

The following format is needed to present aninitial simplex from a file:

[Vertex]ne~[Worst vertex] +F*(Centroid-[Worst vertex])

(Eq. 4)

3

:Calc R : <-----1

II

II

I <-- ICaic R from N : <-----2III II

II II II * II *II .BT.B II II .BT.N II II .BT.W II

II II II II II II

II

:Calc E I

III

II

* II

II .BT.R * ------- IMake R I* * ---------

*

IMake E I

*

11II

II

II

IMake PC I

II

** *

II .BT.W II ->2II *

*

IMake NC I

1I

:*

* *II .BT.W * ->2

* **

*RSP.BT.* --> INew initial simplex: ->1II *

*:*

* *II SD<TollI -> 1

* •*II

** *

*RMS<Tol* ->1

* *l!

IEstab B II Stop I

B,N,WR,E, PC, NC

.BT.RSP

SDTalRMS

CalcMake

best, next-to-worst, worst response, respectively;reflection, expansion, positive and negativecontraction, respectively;better than; yes: down, no: right;mean of responses at all vertices; in this case .BT.means better than previous;standard deviation of all responses;tolerance, typically 0.0001;root mean square deviation from response at thecenter of gravity;calculate coordinates;substitute vertex of worst response by new vertex.

The initial simplex may also be locatedrandomly within a certain range of parameter values.Aberg and Gustavsson (1982) used this technique whenchecking the performance of their simplex algorithm.This concept was expanded to N individually scaledparameters as follows:

123456789012345678PAR

3135

234

l36

333

<--- Identi fication as init ialsimplex f He

<--- Number of parameters<--- Parameters in columns

4

This may be used to manually optimizeinstrumental response, chemical reaction, analyticalmethod, etc.

Input of response at a presented setting ofparameters (T[*]) is required. No input (justreturn) displays the status of the simplex and itsresponse (Fig. 4).

This modification deals with the input buffer[HUF(O)] and in this sense is specific to the HP3000 BASIC. Since it involves just a convenience,it is readily modifiable for computers on which theinput buffer may not be accessible in this way.

The program selects N random numbers and scalesthem to the specified ranges. These become thecoordinates of the center of gravity of the initialsimplex.

Coordinates of the first vertex are:

X11 = X1g - D1/2X21 = X2g - D2/3

Xnl = Xng - Dn/ (n+l)

The other vertices are then calculated. If any ofthe vertices is outside the range originallyspecified, a new center of gravity is selected.

The initial simplex is located with its sideparallel to the plane defined by the first (N-1)parameters. Within this plane, the (N-2)­dimensional edge lies p?rallel to the plane definedby the first (N-2) parameters, etc.

The initial 4-dimensional simplex is then(vertices arranged in columns; Xij is the i-thcoordinate of j-th vertex; Di is the initial step onthe i-th coordinate):

3. Curve fitting

The program expects values of the independentvariable in the array Xl [*] and those of thedependent variable in the array Vl[*J. The fittedfunction must be supplied in FNV (Fig. 5). Theprogram then minimizes the sum of squared deviationsbetween VI [* j and the values calculated froIU thefitted function at giveu values of the independentvariable. For example, if the fitted function is

A= A(inf)*(l-exp(-kt),

the fitted parameters A(inf) and k are stored asT[I j and T[2], respectively. The values of A and tare in the arrays VI[*J and Xl[*j (see function 2 inthe following section).

TESTING PROGRAM'S PERFORMANCE

The performance was tested by finding extremesof functions and by fitting one exponentialfunction. The test functions were those used byAberg and Gustavsson (1982):

1. Two-dimensional paraholoid Z = X**2 + y**Z

The modification of the above to n-dimensions can bereadily accomplished. Z= Sum [A - X*(l - exp(-Yt)]**2

Least-squares response function with parametersfrom Deming and Morgan (1973):

XliX21X3'1X41

Xll+DlX21X31X41

(2X1l+D1)/2X21+D2

X31X41

(2Xll+D1)/2(3X21+D2)/3

X31+D3X41

(2Xll+Dl)/2(3X2l+D2)/3(4X31+D3)/4

X41+D4 2.

Region:Minimum:

-lOOO<=X<=lOUO; -100U<=Y<=1000X=O; y=O; z=O

SIMPLEX ALGORITHM

kept streamlined by coding thethe form of "multiline

resemble FORTRANand facilitate modifications and

translations.

Programming input is required in specifying the"optimized" function with which the algorithm isgoing to deal. The multiline function for this taskis FNV. One of three possibilities is normallyencountered:

1. Function minimization

The function is coded in FNV to return value Sas a function of parameters in the array T[*j. Forexample, to find the minimum of the function:

2 = X**2 + Y**2

the definition of FNV is in Fig. 3.

2. Interactive evaluation

Region O<=X <=2; O<=Y <=2Minimum: X=0.404; Y=0.170; 2=0.0036

Data for the least-squares response function (Demingand Morgan 1973):

t A

1.5 0.1093.0 0.1693.0 0.1724.5 0.2104.5 0.2106.0 0.2516.0 0.2559.0 0.3319.0 0.325

12.0 0.32612.0 0.33015.0 0.36215.0 0.38318.0 0.38118.0 U.37224.0 0.42224.0 0.411

5

4200 DEF FNV(P,S,G,T[*],V1[*],X1[*],V[*],X[*,*])4210 REM Calculate function value4220 REM *.*.**.*••*••*******•••*•••••••******************.*****4230 REM EVALUATED FUNCTION COMES IN HERE •4240 REM Must return value in S4250 S=T[1]*'2+T[2]**24260 REM *4270 REM **.*••*.******••********•••••••••********.**••**.**••••4280 RETURN S4290 FNEND

Fig. 3. Function minimization.

4200 DEF FNV(P,S,G,T[*],V1[1],X1[*],V[*],X[*,*])4210 REM Calculate function value4220 INTEGER I4230 REM 1***•••••••••••••••••••••••••••••••••••••••••••••••••••4240 REM EVALUATED FUNCTION COMES IN HERE •4250 REM Must return value in S4260 FOR 1=1 TO G4270 PRINT T[I]4280 NEXT I4290 INPUT "The response is (CR displays status) ":4300 X=BUF(O)4310 IF X<>4 THEN 18854320 MAT PRINT X4330 PRINT ,,--------"4340 MAT PRINT V4350 GOTO 18404360 REM •4370 REM *.* ••••••••••••••••••••••*•••••••••••••••••••••••••••••4380 INPUT :S:4390 RETURN S4395 FNEND

Fig. 4. Interactive simplex evaluation.

4200 DEF FNV(P,S,G,T[*],V1[*],X1[*],V[*],X[',*])o Calculate function value

4220 INTEGER I4230 3=0

4250 REM EVALUATED FUNCTION COMES IN HERE •4260 REM Must return value in S4270 FOR 1=1 TO P4280 S=S+(V1[I]-T[1]*(1-EXP(-T[2]'X1[I]»)*'24290 NEXT I4300 REM •4310 REM *••*•••••*.*.*.**••••**.*.****.*****•••*.*.*••••••••*••4320 RETURN S4330 FNEND

Fig. 5. Curve fitting.

3. Rosenbrock's function 2(1-X)**2

100*(Y-X)**2 +

6

5. 2

In addition, the following func tion was used:

2*X**3 - 3*X**2 - 2*Y**3 + 3*Y**2R"gion -25<=X<=25; -25<=Y<:=25Minimum: X=l; Y=l; 2=0

4. Modified Fletcher and Powell's helical valley

2 = 100*[X - 10*Q]**2 +[(X**2+Y**2)**.5 -1 ]**2 +Y**2

where

Q=[ATN(Y/X)] /2*Pi for X:>OQ=[Pi + ATN(Y/X)]/2*Pi for X<:O

Region -100<=X<=100; -100<:=Y<:=100Minimum: X=0.691; Y=O.319; 2=0.159

Region -2<=X<=2; -2<=Y<=2Minimum X=l; Y=O; 2=-1Maximum X=O; Y=l; 2=1

The algorithm performed ~ll on all the testfunctions (Table 1). The number of evaluationsneeded for convergence almost doubled wi thdecreasing initial simplex size (ISS). Thisagrees with the results of Aberg and Gustavsson(1982). The convergence was good, even for functionnumber 5. The extremes of this function are"local". Since the optimization was performedwi thout constraints on the variables, thenonconverging simplexes ~re those that failed "tohome in" on the local extremes.

Table 1. Resul ts of tests on func tions specified in text.

---------------------------------------------------------------ISS Paraboloid (1 ) Least squares (2 ) Rosenbrock (3 )

Conv.* Evaluation Conv.* Evaluation Conv.* Evaluation---------- ----------- ----------

Mean SD Mean SD Mean SD----------------------------------------------------------------0.2 0/0/10 36.5 3.3 0/0/10 17.8 2.9 0/0/10 32.2 3.10.1 0/0/10 34.4 3.1 0/0/10 19.5 9.4 0/0/10 31.1 3.80.05 0/0/10 40.9 9.0 0/0/10 25.7 8.8 0/0/10 32.5 4.60.02 0/0/10 37.8 5.4 0/0 /10 24.9 9.3 0/0/10 35.4 5.10.01 0/0/10 38.6 6.1 0/0 /10 24.9 10 .6 0/0/10 36.6 7.80.005 0/0 /10 48.6 3.1 0/1/9 38.8 11. 6 0/0/10 38.2 9.90.001 0/0/10 46.7 3.7 0/1/9 35.7 11.6 0/0/10 47.5 6.80.0005 0/0/10 54.8 7.1 0/2/9 39.2 12.0 0/0/10 54.0 5.8-----------------------------------------------------------------

ISS Helical valley( 4) Minimum (5 ) Maximum (5 )Conv.* Evaluation Conv.* Evaluation Conv.* Evaluation

---------- ----------- ----------Mean SD Mean SD Mean SD

----------------------------------------------------------------0.2 0/0 /10 50.0 6.0 6/0/9 19.3 1.8 5/0/10 18.0 2.20.1 0/0/10 .7 8. 10/0/9 18.2 2.4 8/0/9 17.0 2.40.05 0/0/10 45.5 3. 7/0/9 19. 1+ 3.8 0/0/10 19.6 2.70.02 0/0/10 57.3 15.2 3/0 /10 19.2 3.3 9/0/9 21.8 3.0

0.005 0/1/9 63.3 18.5 12/0 /8 25.4 4.7 9/0/9 27.0 7.50.001 0/0/10 64.2 15.5 13 /1 /8 31.1 7.2 5/0/10 33.6 4.5O. 0005 2/1/9 64.3 7.5 23/0 /7 34.3 8.2 3/0/10 37.8 6.9-----------------------------------------------------------------

ISS = Initial simplex size as a fraction of rangeConv.= Number of: Not converged and detected/Not converged and

not detected/Converged

ACKNOWLEDGHENTS

I thank Mrs. J. Hurley for formatting themanusctipt, Mrs. R. Garnett for editorialassistance, Mrs. M. M. Irwin for proofreading, Mr.S. Bellis for explaining HP-manuals, the staff ofthe computer center for reliable maintenance of thesystem, and Dr. K. Haya and Mr. L. E. Burridge forcomments.

REFERENCES

Aberg, E. R., and A. G. T. Gustavsson. 1982.Design and evaluation of modified simplexmethods. Anal. Chim. Acta 144: 39-53.

Backman, L., and V. P. Shanbag. 1984. Simplexoptimization in biochemistry: Application ofthe method in two-phase part it ion. Anal.Biochem. 138: 372-379.

Berridge, J. C. 1984. Automated multiparameteroptimisation of high-performance liquidchromatographic separations using thesequential simplex procedure. Analyst 109:291-293.

Brissey, G. F., R. B. Spencer, and C. L. Wilkins.1979. High-speed algorithm for simplexoptimization calculations. Anal. Chem.51: 2295-2297.

Brookes, A. E., J. J. Leary, and D. W. Golightly.1981. Modifications of the sequential simplexoptimization method. Anal. Chern. 53: 720-721.

Burgess, D. D., and P. Hayumbu. 1984. Simplexoptimization by advance prediction forsingle-element instrumental neutron activationanalysis. Anal. Chem. 56: 1440-1443.

Cela, R., and J. A. Perez-Bustamante. 1983.Resolution of overlapping peaks in HPLC bymeans of the simplex algorithm. CAL 2:137-144.

Darvas, F., L. Kovacs, and A. Eory. 1978. Computeroptimization by the sequential simplex methodin designing drug analogs. In: QuantitativeStructure-Activity Analysis, R. Franke andP. Oehme, Editors, Abhandlungeg der Akademie

DDR, Nr.2N, 11-315.

Dean, W. K., K. J. Heald, and S. N. Deming. 1975.

Science 189: 805-807.

Deming S. N., and S. L. Morgan. 1973. Simplexoptimization of variables in analyticalchemistry. Anal. Chern. 45: 278A-283A.

IMSL. 1982. IMSL Library Reference Manual. IMSL,Inc. Houston, Texas.

Kaberline, S. L., and C. L. Wilkins. 1978.Evaluation of the super-modified simplex foruse in chemical pattern recognition.Anal. Chim. Acta 103: 417-428.

Keefer, D. L. 1973. Simpat: self-bounding directsearch method for optimization. Ind. Eng. Chem.Process Des. Develop. 12: 92-99.

7

Leggett, D. J. 1983. Instrumental simplexoptimization. J. Chern. Ed. 6U: 707-71U.

Morgan, S. L., and S. N. Deming. 1974. Simplexoptimization of analytical chemical methods.Anal. Chern. 46: 117U-1181.

Nie, N. H., C. H. Hull, J. J. Jenkins, K.Steibrenner, and D. H. Bent. 1975. SPSSStatlstical Package for the Social Sciences.Second Edition. McGraw-Hill Book Company,New York.

O'Neill, R. 1971. Algorithm AS47. Functionminimization using a simplex procedure.Appl. Stat. 20: 338-345.

Olsson, D. M. 1974. A sequential simplex programfor solving minimization problems. J. QualityTechnol. 6: 53-57.

Ritter, G. L., S. R. Lowry, C. L. Wilkins, and T. L.Isenhour. 1975. Simplex patternrecognition. Anal. Chem. 47: 1951-1956.

Routh, M. W., P. A. Schwartz, and M. B. Denton.1977. Performance of the Super ModifiedSimplex. Anal. Chern. 49: 1422-142~.

Ryan, P. B., R. L Barr, and n. D. Todd. 1980.Simplex techniques for nonlinear optimization.Anal. Chern. 52: 1460-1467.

Schnute, J. 1982. A manual of easy nonlinearparameter estimation in fishery research withinteractive microcomputer programs. Can. Tech.Rep. Fish. Aquat. Sci. 1140: 115 p.

Shavers, C. L., M. L. Parsons, and S. N. Deming.1979. Simplex optimization of chemicalsystems. .J. Chern. Ed. 56: 307-309.

Tektronix Inc. 1975. PLOT 50 Statistics Vol.4.3.1-3.9.

van der Wiel, P. F. A., R. Maassen, and G. Kateman.1983. The symmetry-controlled simplexoptimization procedure. Anal. Chim. Acta153: 83-92.

Yamaoka, K., Y. Tanigawara, T. Nakagawa, and T.Uno. 1981. A pharmacokinetic analysis program(MULTI) for microcomputer. J. Pharm. Dyn.I,: 879-885.

ZitkO, V. 1982. Two-compartment model of uptakeand excretion of chemicals. Can. Tech. Rep.

8

Appendix 1 -- Listing of Program SX.

10 FILES 11,11

20 X=RND(O)30 DIM R1[20],R2[20],R3[20],R4[20],XO[20,21]40 DIM X[20,21],V[21],T[20],Tl[20],Wl[20],C[20],G2[20]50 DIM D[5],Dl[4]60 REM X[]-parameters,V[]-SUSQ values, T[],Tl[]-temporary70 REM Wl[]-worst, C[]-centroid,G2[]-center of gravity80 REM D[]- mean response, Dl[ ]-mean response difference, "stacks"90 DIM Xl[50],Vl[50]

100 REM Fitted function indep variable values X1[],110 REM dep variable values Vl []120 DIM N$[8],Nl$[8],N2$[8],N3$[8],B$[80],B1$[20]130 REM N$,Nl$,N2$,N3$-file names; B$,B1$- ASCII buffers140 DIM S9[20,22]150 REM S9[] is summary table160 INTEGER C,CO,C2,C3,C5,I,IO,I5,I9,J,J5,K,G,Gl,P,Q1,Q2,Q3,Q9,Z1170 REAL S,Z,C1,W,N,B180 LONG D190 CO=C2=C4=0200 C3=1210 MAT D=ZER220 HAT D1=ZER230 REM INPUT "Minimum (1), maximum (2) ",C5240 C5=1250 INPUT "Desired function in FNV? (YIN). IF NOT, c-Y or BRK ",Q$260 REM INPUT "Number of evaluations ",10270 10=120280 REM INPUT "Convergence criterium (Le. 1E-4) ",e1290 C1=.0001300 REM INPUT "Print results every n-th evaluation ",17310 17=20320 INPUT "Data from file (1), manually (2), not needed (3) ",Q1330 IF Q1=1 THEN DO340 INPUT "Input file ",Nl$350 ASSIGN Nl$,1,09360 GOSUB 4420370 REDIM Vl[P],X1[P]380 DOEND390 IF Q1=2 THEN DO400 INPUT "Number of points n,p410 REDIM Xl[P],V1[P]420 FOR I= TO P430 INPUT ,Xl[ I]440 INPUT "y ",V1[I]450 NEXT

o DOEND=3

MAT V1=ZER

510 DOEND520 INPUT "Initial simplex from file( 1) ,manually (2), computerO) ", Q1530 IF Q1=1 THEN DO540 INPUT "Initial parameters file ", N2$550 ASSIGN N2$,1,Q9560 GOSUB 4420570 DOEND

9

580 IF Q1<>1 THEN DO590 INPUT "Number of parameters ", G600 DOEND610 G1=G+1620 REDIM R1[G],R2[G],R3[G],R4[G],XO[G,Gl]630 REDIM X[G,G1],V[G1],T[G],Tl[G],Wl[G],C[G],G2[G]640 IF Q1=2 THEN DO650 FOR 1=1 TO Gl660 PRINT "Set of parameters ";1670 FOR J=1 TO G680 PRINT J;690 INPUT X[J,I]700 NEXT J710 NEXT I720 DOEND730 IF Q1=3 THEN 760740 GOSUB 1520750 END760 INPUT "Initial simplexes to file? YIN ",Q$770 IF Q$="N" THEN 850780 INPUT "Filename ", N2$790 CREATE M,N2$,SOO800 IF M=O THEN 820810 GOTO 780820 ASSIGN N2$,2,M830 IF M=O OR M=1 THEN 850840 PRINT "No output to file"850 REM INPUT "Number of initial simplexes ",R9860 R9=10870 REDIM S9[R9,G+2]880 MAT S9=ZER890 FOR 1=1 TO G900 PRINT "Parameter ",I910 INPUT "Minimum ", R1[ I]920 INPUT "Maximum ",R2[I]930 R3[I]=R2[I]-Rl[I]940 INPUT "Initial simplex size as fraction of range ",R4[I]950 NEXT I960 FOR R=1 TO R9970 PRINT "Initial simplex ";R;" *********"980 c=o990 C=C+1

1000 PRINT "Attempt ";C1010 FOR 15=1 TO G1020 X[IS,I]=(R3[I5]*RND(0)+Rl[I5]-R4[IS])/(I5+1)1030 NEXT 151040 FOR 15=1 TO G1050 FOR JS=1 TO IS1060 X[I5,J5]=X[IS,I]

1080 X[I5,I5+1]=X[I5,1]+R3[I5]*R4[I5]1090 NEXT IS1100 FOR 15=1 TO G-l1110 X[IS,I5+2]=01120 FOR JS=1 TO 15+11130 X[I5,I5+2]=X[I5,I5+2]+X[IS,JS]1140 NEXT J5

10

1150 X[15,15+2]=X[15,I5+2]/(I5+1)1160 FOR K=I5+3 TO G+11170 X[I5,K]=X[I5,I5+2]1180 NEXT K1190 NEXT 151200 FOR 15=1 TO G1210 FOR J5=1 TO 0+11220 IF X[I5,J5]<R1[I5] OR X[I5,J5]>R2[I5] THEN 9901230 NEXT J51240 NEXT 151250 MAT PRINT X1260 MAT XO=X121 0 GOSUB 15201280 IF' Q$="Y" THEN DO1290 IF' CO= 1 THEN M$= "N"1300 IF CO=O THEN M$="CIf1310 GOSUB 48701320 DOEND1330 IF CO=1 THEN DO1340 Co=o1350 C4=C4+11360 PRINT "Restart"1370 C2=C2+11380 IF C2>5 THEN 14001390 GOTO 10101400 C2=01410 DOEND1420 NEXT R1430 PRINT1440 IF Q1=1 OR Q1=2 THEN 14901450 PRINT IfIlHlI!ill SUMMARY lll!llUlI"

1460 PRINT "Parameters, Minimum, No of evaluations"1470 MAT PRINT S911180 PRINT "Not converged "; C41490 PRINT IIlllllllllllllllllllllllllllllllllllllllll"

1500 END1510 REM ========================= SIMPLEX ===============1520 19=01530 REM Calculate center of gravity (used instead of centroid)1540 MAT G2=ZER1550 FOR 1=1 TO G1560 FOR J=1 TO 0'11570 a2[I]=a2[I]+X[I,J]1580 NEXT J1590 NEXT I1600

1016201630

1650 P, a,T[1l],V1[1l],X1[1l],V[1l],X[1l,1l])1660 V[I]=S1670 NEXT I1680 Z=FNS(C5,G1,W,N,B,V[1l])1690 19=19+11700 REM F=1 is for reflection1710 F=1

Ll

1720 Z=FNF(G,G1,W,F,G2[*],T[*],X[*,*])1730 MAT T1=T1740 Z=FNV(P,S,G,T[*],V1[*],X1[*],V[*],X[*,*])1750 S1=S1760 IF 19>10 THEN GOTO 49201770 GOTO C5 OF 1780,19101780 REM if better than best look at expansion1790 IF S<V[B] THEN 22301800 REM if better than next-to-worst,substitute for worst1810 GOTO C3 OF 1820,18401820 IF S<V[N] THEN 23301830 GOTO 18501840 IF S<V[W] THEN 23301850 REM if better than worst look at positive contraction1860 GOTO C3 OF 1870,18901870 IF S<V[W] THEN 24301880 GOTO 20301890 IF S<V[N] THEN 24301900 GOTO 20301910 REM if better than best look at expansion1920 IF S>V[B] THEN 22301930 REM if better than next-to-worst,substitute for worst1940 GOTO C3 OF 1950,19701950 IF S>V[N] THEN 23301960 GOTO 19901970 IF S>V[W] THEN 23301980 REM if better than worst look at positive contraction1990 GO TO C3 OF 2000,20202000 IF S>V[W] THEN 24302010 GOTO 20302020 IF S>V[N] THEN 24302030 REM F=-.5 for negative contraction2040 F=-.52050 Z=FNF(G,G1,W,F,G2[*],T[*],X[*,*])2060 Z=FNV(P,S,G,T[*],V1[*],X1[*],V[*],X[*,*])2070 REM if no better than worst reflect from N2080 GOTO C5 OF 2090,21402090 GOTO C3 OF 2100,21202100 IF S>V[W] THEN 2670211 0 GOTO 21 802120 IF S>V[N] THEN 26702130 GOTO 21802140 GOTO C3 OF 2150,21702150 IF S<V[W] THEN 26702160 GOTO 21802170 IF S<V[N] THEN 2670

180 Z=FNR(G,G1,W,B,S,G2[*],T[*],V[*j,X[*,*])2190 PRINT "NC2200 REM reflect from next-to-worst2210 GOTO

2230 F=22240 Z=FNF(G,G1,W,F,G2[*],T[*],X[*,*])2250 Z=FNV(P,S,G,T[*],V1[*],X1[*],V[*],X[*,*])2260 REM if expansion bet tel' than reflection, make it2270 GOTO C5 OF 2280,23002280 IF S<S1 THEN 2380

12

2290 GOTO 23102300 IF S>S1 THEN 23802310 S=S12320 REM make reflection and go to convergence test2330 Z=FNR(G,G1,W,B,S,G2[*],T1[*],V[*],X[*,*])2340 PRINT "R ";2350 GOTO 27902360 REM --------------------------------------------2370 REM make expansion and go to convergence test2380 Z=FNR(G,G1,W,B,S,G2[*],T[*],V[*],X[*,*])2390 PRINT "E ";2400 GOTO 27902410 REM ---------------------------------------------2420 REM F=.5 for positive contraction21130 F=.52440 Z=FNF(G,G1,W,F,G2[*],T[*],X[*,*])2450 Z=FNV(P,S,G,T[*],V1[*],X1[*],V[*],X[*,*])2460 REM if better than next-to-worst, make it2470 GOTO C5 OF 2480,25302480 GOTO C3 OF 2490,25102490 IF S<V[N] THEN 26202500 GOTO 25"(02510 IF S<V[W] THEN 26202520 GOTO 25702530 GOTO C3 OF 2540,25602540 IF S>V[N] THEN 26202550 GOTO 25702560 IF S>V[W] THEN 26202570 REM make it anyway and go to reflection from next-to-worst2580 Z=FNR(G,G1,W,B,S,G2[*],T[*],V[*],X[*,*])2590 PRINT "pit;2600 GOTO 27 102610 REM ---------------------------------------------2620 Z=FNR(G,G1,W,B,S,G2[*],T[*],V[*],X[*,*])2630 PRINT "PC It;2640 GOTO 27902650 REM ----------------------------------------------2660 REM Going to reflect from N but first replacing Wregardless2670 Z=FNR(G,G1,W,B,S,G2[*],T[*],V[*],X[*,*])2680 PRINT liN";2690 REM ----------------------------------------------2700 REM l'eflection from next-to-worst2710 N1=N2720273027402750 C3=2

2770 REM ---_.------------------------------------------2780 REM oonvergence check2790 Z=FNT(C5 ,G1 , V[ 1I] ,D[lI] ,D1[ II])2800 FOR 1=1 TO #2810 IF ABS(D1[I]».00001 THEN 28802820 GOTO C5 OF 2830,28502830 IF D1[I]>0 THEN 49202840 GOTO 28602850 IF D1[I]<0 THEN 4920

l3

2860 NEXT I2870 GOTO 29902880 C3=12890 Z=FND(G,G1,V[*],D)2900 IF 19>10 THEN GOTO 49202910 IF 19 MOD 17=0 THEN DO2920 PRINT2930 PRINT"# ";192940 GOSUB 48102950 DOEND2960 IF D>C1 THEN 16802970 Z=FNG(G,G1,V[*],X[*,*],G2[*],D)2980 IF D>C1 THEN 16802990 Z=FNS(C5,G1,W,N,B,V[*])3000 PRINT3010 PRINT" ***********> CONVERGED <*************"3020 PRINT "After "; 19;" function evalua tions"3030 PRINT "Optimized parameters"3040 PRINT "Best"3050 H=B3060 GOSUB 116003070 PRINT "Worst 11

3080 H=W3090 GOSUB 116003100 PRINT "------------"311 0 GOSUB 481 03120 IF Q1=1 OR Q1=2 THEN 31803130 FOR 1=1 TO G3140 S9[R,I]=X[I,B]3150 NEXT I3160 S9[R,G+1]=V[B]3170 S9[R,G+2]=I93180 RETURN3190 REM ============= Functions definitions ===========3200 DEF FND(G,G1,V[*],LONG D)3210 REM Standard deviation of centroid vertices in conv.test3220 INTEGER I3230 REAL 31,S23240 S1=S2=03250 FOR 1=1 TO G13260 S1=S1+V[I]3270 32=32+V[I]*V[I]3280 NEXT I3290 D=SQR(ABS(((01) )/(0*G1)))3300 RETURN D3310 FNEND

3330334033503360337033803390340034103420

REM Find coordinates, type depending on FINTEGER IFOR 1=1 TO G

T[I]=G1*(1+F)*G2[I]/G-(1+G1*F)*X[I,W]/GNEXT IRETURN T[ 1]

FNENDDEF FNG(G,G1,V[*],X[*,*],G2[*],LONG D)

REM Find response in the center of gravityREM from vertices

& its deviation

14

3430 INTEGER I3440 REAL S3450 Z=FNV(P,S,G,G2[*],V1[*],X1[*],V[*],X[*,*])3460 D=O3470 FOR 1=1 TO G13480 D=D+(V[I]-S)*(V[I]-S)31190 NEXT I3500 D=SQR(ABS(D/G1))3510 RETURN D3520 FNEND3530 DEF FNR(G,G1,W,B,S,G2[*],T[1l],V[*],X[*,*])3540 REM Replace worst vertex by coordinates in T[*]3550 REM and update center of gravity3560 INTEGER I3570 FOR 1=1 TO G3580 G2[I]=G2[I]+(T[I]-X[I,W])/G13590 X[I,W]=T[I]3600 NEXT I3610 V[W]=S3620 B=W3630 RETURN X[1,1]3640 FNEND3650 DEF FNS(C5,G1,W,N,B,V[*])3660 REM Sort responses and find Worst, Next-to-worst, and Best3670 INTEGER I3680 REAL M1,M23690 M1=-1E703700 M2=1E703710 FOR 1=1 TO G13720 IF V[I]<M1 THEN 37503730 M1=V[I]3740 W=I3750 IF V[I]>M2 THEN 37803760 M2=V[I]3770 B=I3780 NEXT I3790 IF C5=2 THEN DO3800 N=W3810 W=B3820 Bdl3830 DOEND3840 GOTO C5 OF 3850,39303850 M1=-1E703860 FOR 1::1 TO G13870 IF I=W THEN 39103880 V[ I THEN 39103890 ::V[I3900 N=I

3920 GOTO 40003930 M2=1E703940 FOR 1=1 TO G13950 IF I=W THEN 39903960 IF V[I]>M2 THEN 39903970 M2=V[I]3980 N=I3990 NEXT I

LS

4000 RETURN W4010 FNEND4020 DEF FNT(C5,Gl,V[*],D[*],D1[*])4030 REM Mean response and its difference "stack"40 110 INTEGER I4050 D[1]=D[2]4060 D[2]=D[3]4070 D[3]=D[4]4080 D[4]=D[5]4090 D[5]=04100 FOR 1=1 TO Gl4110 D[5]=D[5]+V[I]4120 NEXT I4130 D[5]=D[5]/G14140 Dl[ 1]=D[2]-D[ 1]4150 Dl[2]=D[3]-D[2]4160 D1[3]=D[4]-D[3]4170 Dl[4]=D[5]-D[4]11180 RETURN D[ 1]4190 FNEND4200 DEF FNV(P,3,G,T[*],V1[*j,X1[*],V[*],X[*,*])4210 REM Calculate function value4220 INTEGER I4230 3=04240 REM .******************************************************4250 REM EVALUATED FUNCTION COME3 IN HERE *4260 REM Must return value in 311270 REM FOR 1= 1 TO Plj280 REM 3=3+(Vl[I]-T[1]1I(1-EXP(-T[2]*X1[IJ)))**211290 REM NEXT I4300 REM 3=T[1]*T[1]+T[2]*T[2]4310 REM 3=100*(T[2]-T[1])*(T[2]-T[1])+(1-T[1])*(1-T[1])4320 REM IF T[1]>0 THEN Q=ATN(T[2]/T[1])/6.283184330 REM IF T[1]<0 THEN Q=(6.28318+ATN(T[2]/T[1]))/6.283184340 REM 3=100*(T[1]-10*Q)**2+«T[1]**2+T[2]**2)**.5 -1)**24350 REM 3=S+T[2]ll!!24360 3=2*T[1]**3-3*T[1]**2-2*T[2]**3+3*T[2]**211370 REM S=2-(T[1]-1)!Ill2-(T[2]-3) 1Hl 24380 REM *******************.*•••••••••••••••*•••••••••••*••*•••4390 HETURN 34400 FNEND4410 REM ===============1/0 Subroutines =======================4420 IF Q9=0 THEN lj4504430 PRINT "File problem"4440 STOP11450 it1 ; B$4460 B1$=B$[1;3]

4480 REM Initial parameter file4490 B1$=DEB$(B1$)4500 CONVERT B1$ TO P4510 FOR 1=1 TO P4520 LIN PUT #1;B$4530 B1 $=B$[ 1; 14]4540 CONVERT B1$ TO X1[I]4550 B1$=B$[15;14]4560 CONVERT B1$ TO V1[I]

16

4570 NEXT I4580 RETURN4590 REM Output subroutine4600 FOR 1=1 TO G4610 PRINT X[I,H]j4620 NEXT I4630 PRINT4640 PRINT "Optimized value "jV[H]4650 RETURN4660 LINPUT #1;B$4670 B1$=B$[1;2]4680 B1$=DEB$(B1$)4690 CONVERT B1$ TO G4700 G1=G+14710 FOR 1=1 TO G4720 LIN PUT #1jB$4730 FOR J=1 TO G+14740 J1=5*J-44750 IF J1>65 THEN J1=J1-644760 B1$=B$[J1j5]4770 CONVERT B1$ TO X[I,J]4780 NEXT J4790 NEXT I4800 RETURN4810 PRINT4820 MAT PRINT X4830 PRINT "--------,,48110 MAT PRINT V4850 RETURN4860 REM -----------------------------------------4870 PRINT #2jM$4880 MAT PRINT #2;XO4890 PRINT #2;V[B]4900 RETURN4910 REM ------------------------------------4920 PRINT4930 PRINT "Not converged after "jI9;" function evaluations"4940 GOSUB 1181011950 CO=14960 GOTO 3180

SAVE SX!EXIT

17

Appendix 2 - Example of interactive input of parameters and output of intermediate results.

:RUN SXC

Desired function in FNV? (YIN). IF NOT, c-Y or BRK ]Data from file (1), manually (2), not needed (3) 3Initial simplex from file(1) ,manually (2),computer(3) 3Number of parameters 2Initial simplexes to file? YIN NParameter 1Minimum -100Maximum 100Initial simplex size as fraction of range .05Parameter 2Minimum -100Maximum 100Initial simplex size as fraction of range .05Initial simplex 1 *********Attempt 1

42.5637 52.5637 47.5637

27.2918 27.2918 37.2918

E E R PC NC Prn Prn NC E NC R ERE NC R NC NC NC Nrn R E NC NC NC R Nrn NC Nrn NC R NC R NC NC ERE NC Nrn R R NC R NC R NC R NC ERR R NCR Nrn NC NC NC R# 60

.692031

.321541

.159523

.6953

.323852

.159401

.683329

.312119

.1595

.323852.159401

NC NCa***.******> CONVERGED <*************

After 62 function evaluationsOptimized parametersBest

.688164 .316951Minimized value .159345Worst

.6953Minimized

****** SUMMARY ******Parameters, Minimum, No of evaluations

.688164 .316951 .159345 62

18

.688512 .317 428 .159341 48

.687781 .316732 .159354 56

.694594 .323637 .159401 59

.691574 .32074 .159379 59

.69364 .322438 .15935 51

.694736 .323439 .159374 52

.690261 .318711 .159369 52

.686567 .315556 .159381 61

.69359 .322634 .159379 53

Not converged 0•••••••********.*••*.

END OF PROGRAM