Direct methods Nelder-Mead

There are two basic types of unconstrained optimization algorithms (I) those reqmring function derivatives and (2) those that do not. The nonderivative methods are of interest in optimization applications because these methods can be readily adapted to the case in which experiments are carried out directly on the process. In such cases, an ac tual process measurement (such as yield) can be the objec tive function, and no mathematical model for the process is required. Methods that do not reqmre derivatives are called direc t methods and include sequential simplex (Nelder-Meade) and Powell s method. The sequential simplex method is quite satisfac tory for optimization with two or three independent variables, is simple to understand, and is fairly easy to execute. Powell s method is more efficient than the simplex method and is based on the concept of conjugate search directions. 

Basically two search procedures for non-linear parameter estimation applications apply. (Nash and Walker-Smith, 1987). The first of these is derived from Newton s gradient method and numerous improvements on this method have been developed. The second method uses direct search techniques, one of which, the Nelder-Mead search algorithm, is derived from a simplex-like approach. Many of these methods are part of important mathematical computer-based program packages (e.g., IMSL, BMDP, MATLAB) or are available through other important mathematical program packages (e.g., IMSL). 

If the numerical computation of the gradient of an objective function shall be avoided, and if accuracy requirements are not too high, a direct method such as the Nelder-Mead simplex algorithm [12] implemented in the Scilab function fminsearch () may be used that allows for noise in the cost function. 

Various more-or-less efficient optimization strategies have been developed [46, 47] and can be classified into direct search methods and gradient methods. The direct search methods, like those of Powell [48], of Rosenbrock and Storey [49] and of Nelder and Mead ( Simplex ) [50] start from initial guesses and vary the parameter values individually or combined thereby searching for the direction to the minimum SSR. 

Many methods are available for least-squares calculations. Models linear in 6 allow direct solutions other models need iteration. The choice of iteration method depends on one s goal. For a mere curve-fit of the data, a direct search procedure such as that of Powell (1965) or of Nelder and Mead (1965) may suffice. But to determine the most important parameters and their most probable values, a method based on derivatives of S 0) is essential and is followed here. 

Numerous modifications have been made to the original simplex method. One of the more important modifications was made by Nelder and Mead l who modified the method to allow expansions in directions which are favorable and contractions in directions which are unfavorable. This modification increased the rate at which the optimum is found. Other important modifications were made by Brissey l who describes a high speed algorithm, and Keefert" who describes a high speed algorithm and methods dealing with bounds on the independent variables. 

The classic example of a direct search algorithm is the simplex method of Nelder and Mead (1965), who utilized a method originally devised by Spendley et al. (1962). In p-dimensional space, a simplex is a polyhedron of p + 1 equidistant points forming the vertices. For a two-dimensional problem, the simplex is an equilateral triangle. For three-dimensions, the simplex is a tetrahedron. This algorithm, which has no relation to... 

The steep concentration and temperatui-e profiles in the integral reactor did not allow to determine the reaction rates imm.ediately. Therefore, the objective function contains the measured and the calculated concentrations instead of the reaction rates, also the temperatures because of the nonisothermal reactor behaviour. The kinetic parameters must be obtained by direct search techniques like the derivative free simplex method of Nelder and Mead. 

---