Nelder-Mead modification

Note that in the original version by Nelder-Mead, the vertex vr was accepted when it was better than vq but not necessarily better than vr. Nowadays, it is standard form to introduce this modification. 

The rigidity that prevented an accurate optimal point from being obtained was solved by Nelder and Mead [17] in 1965. They proposed a modification of the algorithm that allowed the size of the simplex to be varied to adapt it to the experimental response. It expanded when the experimental result was far of the optimum - to reach it with more rapidly - and it contracted when it approached a maximum value, so as to detect its position more accurately. This algorithm was termed the modifiedsimplex method. Deming and it co-workers published the method in the journal Analytical Chemistry and in 1991 they published a book on this method and its applications. 

By far the most popular technique is based on simplex methods. Since its development around 1940 by DANTZIG [1951] the simplex method has been widely used and continually modified. BOX and WILSON [1951] introduced the method in experimental optimization. Currently the modified simplex method by NELDER and MEAD [1965], based on the simplex method of SPENDLEY et al. [1962], is recognized as a standard technique. In analytical chemistry other modifications are known, e.g. the super modified simplex [ROUTH et al., 1977], the controlled weighted centroid , the orthogonal jump weighted centroid [RYAN et al., 1980], and the modified super modified simplex [VAN DERWIEL et al., 1983]. CAVE [1986] dealt with boundary conditions which may, in practice, limit optimization procedures. 

A simplex is a convex geometric figure of k+1 non-planar vertices in k dimensional space, the number of dimensions corresponding to the number of independent factors. Thus, for two factors, it is a triangle, and for three factors, it is a tetrahedron. The method is sequential because the experiments are analyzed one by one as each is carried out. The basic method used a constant step size, allowing the region of experimentation to move at a constant rate toward the optimum. However, a modification that allows the simplex to expand and contract, proposed by Nelder and Mead in 1965, is more generally used. It has been reviewed recently by Waters. ... 

A modified simplex method is described below. The procedure is close to the modifications suggested by Nelder and Mead.[2a] Other modifications are described in the works given in the reference list. 

Numerous modifications have been made to the original simplex method. One of the more important modifications was made by Nelder and Mead l who modified the method to allow expansions in directions which are favorable and contractions in directions which are unfavorable. This modification increased the rate at which the optimum is found. Other important modifications were made by Brissey l who describes a high speed algorithm, and Keefert" who describes a high speed algorithm and methods dealing with bounds on the independent variables. 

---