The Nelder-Mead Simplex Method

Spendley et al. (1962) proposed the Simplex method that is different to the homonymous method adopted in linear programming (Chapter 10 Vol. 5 - Buzzi-Ferraris and Manenti, in progress). This method was subsequently improved upon by Nelder and Mead (1965). 

Vertices are sorted to have increasing function values with respect to the index of vertices Fq F F -i Fn- The vertex vq contains the best value and v the worst value of the N + 1 vertices. 

The barycentric v of the vertices from 0 to N — 1 is calculated through the arithmetic mean of their coordinates by excluding the worst vertex v.  

Note that in the original version by Nelder-Mead, the vertex vr was accepted when it was better than vq but not necessarily better than vr. Nowadays, it is standard form to introduce this modification. 

The simplex, on the other hand, is contracted in the neighborhood of the best vertex vq by means of the formula  

---

In the book, Vapor-Liquid Equilibrium Data Collection, Gmehling and colleagues (1981), nonlinear regression has been applied to develop several different vapor-liquid equilibria relations suitable for correlating numerous data systems. As an example, p versus xx data for the system water (1) and 1,4 dioxane (2) at 20.00°C are listed in Table El2.3. The Antoine equation coefficients for each component are also shown in Table E12.3. A12 and A21 were calculated by Gmehling and colleaques using the Nelder-Mead simplex method (see Section 6.1.4) to be 2.0656 and 1.6993, respectively. The vapor phase mole fractions, total pressure, and the deviation between predicted and experimental values of the total p... 

The Nelder-Mead simplex method is not really a global search technique, but rather a local method that takes large steps and does not use gradients. Because of these properties, it can be used for global searches in certain cases. 

The Nelder-Mead SIMPLEX algorithm has been frequently used in Analytical Chemistry as well as in other areas of science and engineering. Assessment and further development of the method remains an active field of research (4.). 

The isotherm parameters were determined using Nelder Mead simplex method by minimizing the sum of residual, namely, the differences between experimental and estimated adsorption amount. Figure 2 showed the adsorption isotherms of TCE on MCM-48 at 303, 308, 313, 323 K. As one can be expected, the adsorption capacity was decreased with increasing temperature. The hybrid isotherm model for a pure adsorbate was found to fit the individual isotherm data very well. The parameters of the hybrid equations are listed in Table 1. 

The Nelder-Mead simplex algorithm was published already on 1965, and it has become a classic (Nelder Mead, 1965). Several variants and applications of it have been published since then. It is often also called the flexible polyhedron method. It should be noted that it has nothing to do with the so-called Dantzig s simplex method used in linear programming. It can be used both in mathematical and empirical optimization. 

If the numerical computation of the gradient of an objective function shall be avoided, and if accuracy requirements are not too high, a direct method such as the Nelder-Mead simplex algorithm [12] implemented in the Scilab function fminsearch () may be used that allows for noise in the cost function. 

Jensen et al. demonstrated an integrated microreactor system with online HPLC analysis to optimise the Heck reaction of 4-chlorobenzotrifluoride and 2,3-dihydrofuran (Figure 12.4). The optimisation was controlled using a Nelder-Mead Simplex method, a black-box approach, which required no a priori reaction or gradient information. They demonstrated that this process could be optimised to produce a yield of 83% after 19 experiments, each taking approximately 20 minutes including analysis time. 

Basically two search procedures for non-linear parameter estimation applications apply. (Nash and Walker-Smith, 1987). The first of these is derived from Newton s gradient method and numerous improvements on this method have been developed. The second method uses direct search techniques, one of which, the Nelder-Mead search algorithm, is derived from a simplex-like approach. Many of these methods are part of important mathematical computer-based program packages (e.g., IMSL, BMDP, MATLAB) or are available through other important mathematical program packages (e.g., IMSL). 

A polynomial was fit to the calibration curve for the thermocouple by means of a minimization of the maximum deviation technique using the Nelder Mead sequential simplex minimization algorithm method.( 5,6, 7) The coefficients of this polynomial are stored in the analysis program and are used to convert thermocouple voltages to temperature values. Y values are converted to dH(t,T)/dt, the heat flow into and out of the sample in mcal/sec. The operator selects a baseline for the analysis by entering the temperatures of the beginning and end points of the baseline. A plot is produced of the raw data with the operator selected baseline shown as illustrated in Figure A. 

The key problem of the dissociation model is then the correct evaluation of aot and q o2- In order to solve this problem, the Nelder-Mead (1964) numerical minimization algorithm was used. This algorithm represents an extension of the simplex method of Spendley et al. (1962). The analysis of the validity of the procedure was made using the data of Olteanu and Pavel (1995) for electrical conductivities and molar volumes. 

The Nelder-Mead downhill simplex method is the optimization technique incorporated in the software package Matlab as fin ins or fininsearch. 

The Nelder-Mead downhill simplex algorithm has the advantage of being very reliable and requiring no derivative evaluations. On the other hand, it is slower than methods that do use derivative information. Of course, even a relatively slow method may be quite fast enough for a not-too-large problem on a fairly fast computer. 

The BzzMinimizationSimplex class is designed to solve unconstrained multidimensional minimization problems by means of the Nelder-Mead version of the Simplex method, whereas the BzzMinimizationRobust class uses the Optnov method combined with a robust version of the Simplex method. 

Commercially available software developed to process individual impedance spectra use few general algorithms such as Levenberg-Marquardt algorithm, the Nelder-Mead downhill simplex method or genetic algorithms [3-7]. The software is optimized to process only... 

The steep concentration and temperatui-e profiles in the integral reactor did not allow to determine the reaction rates imm.ediately. Therefore, the objective function contains the measured and the calculated concentrations instead of the reaction rates, also the temperatures because of the nonisothermal reactor behaviour. The kinetic parameters must be obtained by direct search techniques like the derivative free simplex method of Nelder and Mead. 

There are two basic types of unconstrained optimization algorithms (I) those reqmring function derivatives and (2) those that do not. The nonderivative methods are of interest in optimization applications because these methods can be readily adapted to the case in which experiments are carried out directly on the process. In such cases, an ac tual process measurement (such as yield) can be the objec tive function, and no mathematical model for the process is required. Methods that do not reqmre derivatives are called direc t methods and include sequential simplex (Nelder-Meade) and Powell s method. The sequential simplex method is quite satisfac tory for optimization with two or three independent variables, is simple to understand, and is fairly easy to execute. Powell s method is more efficient than the simplex method and is based on the concept of conjugate search directions. 

It is noted that Press et al. (1992) give a subroutine that implements the simplex method of Nelder and Mead. They also recommend to restart the minimization routine at a point where it claims to have found a minimum... 

Nelder and Mead (1965) described a more efficient (but more complex) version of the simplex method that permitted the geometric figures to expand and contract continuously during the search. Their method minimized a function of n variables using (n + 1) vertices of a flexible polyhedron. Details of the method together with a computer code to execute the algorithm can be found in Avriel (1976). 

SIMPLEX method (Nelder and Mead, 1995). Alternative functions for the background and for the peak shapes, furthermore alternative approaches for locating the global minimum are planned to be introduced in later versions of the program. 

Fig. 2.13. Logic diagram of the simplex method of Nelder and Mead...

---