Nelder-Mead simplex procedure

A variety of rules have been developed to control the movement and adaptation of the simplex, of which the most famous set is due to Nelder and Mead (Olsson and Nelson, 1975). The Nelder-Mead simplex procedure has been successfully used for a wide range of optimization problems and, due to its simple implementation, is amongst the most widely used of all optimization techniques. Importantly for the current application, simplex optimization is a black-box technique since it uses only the comparative values of the function at the vertices of the simplex to advance the position of the simplex, and it therefore requires no knowledge of the underlying mathematical function. It is also well suited to the optimization of expensive functions since as few as one new measurement is needed to advance the simplex one step. In its usual form, simplex optimization is suitable only for unconstrained optimization, but effective constrained versions have also been developed (Parsons et al., 2007 ... 

The next problem for nonideal multicomponent mixtures is to solve the n activity coefficients for the x, values at the total surfactant composition and concentration. To solve the n activity coefficients and the n mole fractions, we need 2n equations n equations of (10.33) and n equations of (10.31) or (10.32), with the constraint that the sum of the x, values equals unity. A numerical solution of multiple equations for multiple unknowns can be reached efficiently using the Nelder-Mead simplex technique." Once the y, values have been determined, the mole fraction in micelle x, and the monomer concentration C, for a multicomponent surfactant solution are easily determined by (10.29) and (10.30). The former values are, of course, obtained together with the y, values. Figure 10.4 shows the CMCs determined by this procedure for the ternary mixture of CioH2i(CH3)2PO/CioH2i(CH3)SO/Ci2H25S04Na. For this ternary mixture, fiq. (10.33) is written as... 

Basically two search procedures for non-linear parameter estimation applications apply. (Nash and Walker-Smith, 1987). The first of these is derived from Newton s gradient method and numerous improvements on this method have been developed. The second method uses direct search techniques, one of which, the Nelder-Mead search algorithm, is derived from a simplex-like approach. Many of these methods are part of important mathematical computer-based program packages (e.g., IMSL, BMDP, MATLAB) or are available through other important mathematical program packages (e.g., IMSL). 

To offer more flexibility we adopt an approach, based on the transient simulation model TRNSYS (Klein et al., 1976), making use of the Lund DST borehole model (Hellstrom, 1989). The parameter estimation procedure is carried out using the GenOPT (Wetter, 2004) package with the Nelder and Mead Simplex minimization algorithm (Nelder and Mead, 1965) or Hooke and Jeeves minimization algorithm (Hooke and Jeeves, 1961). 

A more rewarding solution to this problem is the use of modified Simplex procedures, such as first described by Nelder and Mead [507], Such modified algorithms allow other operations besides reflecting the triangle, such as contraction or expansion. The manner in which such a modified Simplex algorithm proceeds is illustrated in figure 5.8 for a... 

The key problem of the dissociation model is then the correct evaluation of aot and q o2- In order to solve this problem, the Nelder-Mead (1964) numerical minimization algorithm was used. This algorithm represents an extension of the simplex method of Spendley et al. (1962). The analysis of the validity of the procedure was made using the data of Olteanu and Pavel (1995) for electrical conductivities and molar volumes. 

In the basic simplex method, the simplex thus can only be reflected to obtain the next experiment, and the simplex size remains the same throughout the procedure. In the modified simplex method, suggested by Nelder and Mead (100), the simplex can be reflected, expanded, or contracted to define the next experiment. Thus, in case the simplex is expanded or contracted, the simplex size changes. More information about the simplex procedures can be found in References 7,9,10, and 98-102. 

Let us now consider the variable-size or modified simplex procedure, proposed by Nelder and Mead (100). Whereas in the basic procedure, the size is fixed and determined by the initially chosen simplex, the size in the modified simplex procedure is variable. Besides the rules of the basic procedure, the modified procedure additionally allows expansion or contraction of simplexes. In favorable search directions, the simplex size is expanded to accelerate finding the optimum, while in other circumstances, the simplex size is contracted, for example, when approaching the optimum (Figure 2.14). 

Based on the analytical expression for the derivative of det[ V(p) ], Bates and Watts (ref. 30) recently proposed a Gauss-Newton type procedure for minimizing the objective function (3.66). We use here, however, the simplex method of Nelder and Mead (module M34) which is certainly less efficient but does not require further programming. The determinant is evaluated by the module M14. After 95 iterations we obtain the results shown in the second row of Table 3.5, in good agreement with the estimates of Box et al. (ref. 29 ). 

By far the most popular technique is based on simplex methods. Since its development around 1940 by DANTZIG [1951] the simplex method has been widely used and continually modified. BOX and WILSON [1951] introduced the method in experimental optimization. Currently the modified simplex method by NELDER and MEAD [1965], based on the simplex method of SPENDLEY et al. [1962], is recognized as a standard technique. In analytical chemistry other modifications are known, e.g. the super modified simplex [ROUTH et al., 1977], the controlled weighted centroid , the orthogonal jump weighted centroid [RYAN et al., 1980], and the modified super modified simplex [VAN DERWIEL et al., 1983]. CAVE [1986] dealt with boundary conditions which may, in practice, limit optimization procedures. 

A modified simplex method is described below. The procedure is close to the modifications suggested by Nelder and Mead.[2a] Other modifications are described in the works given in the reference list. 

---