Simplex variable-size

Progress of a variable-sized simplex optimization for the response surface of Example 14.1. The optimum response is at (3, 7). 

A variable-size simplex optimization of a gas chromatographic separation using oven temperature and carrier gas flow rate as factors is described in this experiment. 

This experiment describes a variable-size simplex optimization of the quantitative analysis of vanadium as... 

In this experiment the goal is to mix solutions of 1 M HCl and 20-ppm methyl violet to give the maximum absorbance at a wavelength of 425 nm (corresponding to a maximum concentration for the acid form of methyl violet). A variable-size simplex optimization is used to find the optimum mixture. 

A simplex is a geometric figure that has one more point than the number of factors. So, for two factors or independent variables, the simplex is represented by a triangle. Once the shape of a simplex has been determined, the method can employ a simplex of fixed size or of variable sizes that are determined by comparing the magnitudes of the responses after each successive calculation. Figure 5 represents the set of... 

Examples of optimizations in HPLC using the simplex approach can be found in [28,84]. In [28] the mobile phase composition for the chiral separation of (6/ )- and (65)-leucovorin on a BSA (bovine serum albumin) stationary phase is optimized by means of a variable-size simplex. Three factors were examined, the pH of the mobile phase buffer, the ionic strength of the buffer and the percentage of 1-propanol in the mobile phase. Table 6.19 shows the experimental origin, the initial step size and the acceptable limits for the factors. The criterion optimized is the valley-to-peak ratio (Section 6.2). The points selected and the results are pre.sented in Table 6.20 and... 

CONSECUTIVE POINTS SELECTED AND MEASURED RESULTS IN THE VARIABLE-SIZE SIMPLEX OPTIMIZATION... 

In [84] a variable size simplex is u.sed to optimize the mobile phase composition in RP chromatography. The fractions of methanol and water in a methanol/water/acetonitrile mobile phase were optimized. The feasible experimental area was determined by carrying out a gradient elution separation and then calculating the boundaries of the variable space in which the simplexes should move to find an optimized isocratic separation. 

It is observed that the simplexes circle around the optimum and point 8 is the closest the real optimum can be reached by the simplex used. The number of experiments or simplexes required to approach the optimum depends on the size of the simplex. A larger simplex will require fewer experiments than a smaller simplex. However, a smaller simplex will allow approaching the real optimum closer than a larger one. From this need to find a compromise between speed of moving through the domain and approachabUity of the optimum, the variable-size or modified simplex procedure has been developed. 

Let us now consider the variable-size or modified simplex procedure, proposed by Nelder and Mead (100). Whereas in the basic procedure, the size is fixed and determined by the initially chosen simplex, the size in the modified simplex procedure is variable. Besides the rules of the basic procedure, the modified procedure additionally allows expansion or contraction of simplexes. In favorable search directions, the simplex size is expanded to accelerate finding the optimum, while in other circumstances, the simplex size is contracted, for example, when approaching the optimum (Figure 2.14). 

With the variable-size simplex, the step width is changed by expansion and contraction of the reflected vertices. The algorithm is modified as follows (cf. Figure 4.18) ... 

In this example, the performance of the variable-size simplex is demonstrated for the enzyme determination based on the problem in Examples 4.4 and 4.7. For a fixed enzyme concentration of 13.6 mg-l ceruloplasmin (coded 0), the concentration of the substrate PPD and the pH value are sought for the maximum rate of the reaction, y. Since the simplex searches for a minimum, the rate as the objective criterion has to be... 

Figure 7.12 Optimization using variable-size simplexes for details, see text.

The sequential simplex method of optimization was proposed by Nelder and Mead. With a number of improvements and enhancements the simplex method has found great utility in real situations in analytical laboratory experiments and process control situations. The simplex method is a hill-climbing method that seeks to climb the response surface depending on the features of the response surface in its immediate neighborhood. Only one new experiment is done for each step in the optimization sequence, and the location of this new experiment on the response surface is completely determined by the previous few experiments. The method of Nelder and Mead using a variable-size simplex is the most commonly used. A complete description of how the simplex method works is beyond the scope of this review, since the information is contained elsewhere.Many applications of simplex optimization have appeared, and a few examples follow. 

Chubb and co-workers applied simplex optimization to increasing the yield of the Bucherer-Berg reaction. In this reaction, a ketone reacts with NH3, COS, and HCN to give a complex, heterocyclic product. Eight variables are related to the yield obtained the initial concentrations of the four reactants, pH, temperature, time of the reaction, and the solvent used. A mixed solvent of ethanol and water was used, with the ratio varied as part of the optimization. Results were reported for several sets of experiments using cyclohexanone and adamantanone as the starting material. The yields were improved rapidly using a variable-size simplex procedure. In one experiment, the yield was improved from 49 to 88%. 

The basic simplex optimization method, first described by Spendley and co-workers in 1962 [ 11 ], is a sequential search technique that is based on the principle of stepwise movement toward the set goal with simultaneous change of several variables. Nelder and Mead [12] presented their modified simplex method, introducing the concepts of contraction and expansion, resulting in a variable size simplex which is more convenient for chromatography optimization. 

A weakness with the standard mediod for simplex optimisation is a dependence on the initial step size, which is defined by the initial conditions. For example, in Figure 2.37 we set a very small step size for both variables this may be fine if we are sure we are near the optimum, but otherwise a bigger triangle would reach the optimum quicker, the problem being that the bigger step size may miss the optimum altogether. Another method is called the modified simplex algorithm and allows the step size to be altered, reduced as the optimum is reached, or increased when far from the optimum. 

To define the first simplex the multiplication factors shown in Table 6.18 are used. Suppose the simplex is a triangle since two variables are being optimized. The experimenter defines a first point (the first vertex, also called the experimental origin) and the step size for each variable, i.e. the maximum change one wants to apply for a variable at each step of the procedure. For instance, for the first point variable, vi = 10 and variable xi = 100 with step sizes of 5 and 10, respectively. The vertices of the initial triangle are obtained as Vertex 1... 

In the simplex procedures described above the step size was fixed. When the step size was taken too small it takes a large number of experiments to reach the optimum, and when it is taken too large the supposed optimum can be unacceptably far from the real one. To avoid this a so-called modified simplex method can be applied, in which the step size is variable throughout the procedure. The principles of the simplex method are maintained but rules for expansion or contraction of the simplexes are added. For a detailed description of these guidelines we refer to [27,831. 

If a point is retained in three consecutive simplexes, then it can be assumed that an optimum has been reached. (Note it may be that this optimum is not the true optimum, but that the simplex has been trapped at a false optimum. In this situation, it is necessary to start the simplex again, or use a modified simplex in which the step size is not fixed but variable, see Fig. 43.3.)... 

This gave much more reliable results. However, we had not implemented a variable step-size algorithm, and therefore obtained fit parameters that were only in the neighborhood of the best fit parameters. Given our previous experience with simplex searching, we combined the two techniques in order to obtain more precise parameter estimates. 

Another approach to obtaining more accurate parameter estimates with simulated annealing analysis is the use of a variable step size algorithm. This was done by Sutter and Kalivas [27]. However, given om previous experience with simplex searching, we elected to combine the techniques of simulated annealing and simplex searching. 

Define a starting value, m of the natural variable, and a step-length for its variation, Su. The step length will define how large a part of the experimental domain is covered by the design, i.e. the "size" of the simplex. 

If prior knowledge of the influence of the variables is available, the size of the simplex can be chosen to give a good progression along the path of the steepest ascent. Close to an optimum, the simplex will encircle the optimum conditions. As these experimental points are evenly distributed, they can be incorporated in a equiradial design (see Chapter 12) which can be used to fit a response surface model in the optimum domain. 

After optimization by simplex, the plot from Figure 12.6 improved and yielded a linear equation (log[p] = -3.14 - 0.20 log[bp], = 0.998) suitable for the analysis of the 201-2036 bp size range. This equation was further used to determine the size of unknown DNA fragments (27). Thus, the simplex method was shown to be an efficient way to optimize an electrophoretic separation of DNA, since several variables could be simultaneously optimized. 

The Matlab Simulink Model was designed to represent the model stmctuie and mass balance equations for SSF and is shown in Fig. 6. Shaded boxes represent the reaction rates, which have been lumped into subsystems. To solve the system of ordinary differential equations (ODEs) and to estimate unknown parameters in the reaction rate equations, the inter ce parameter estimation was used. This program allows the user to decide which parameters to estimate and which type of ODE solver and optimization technique to use. The user imports observed data as it relates to the input, output, or state data of the SimuUnk model. With the imported data as reference, the user can select options for the ODE solver (fixed step/variable step, stiff/non-stiff, tolerance, step size) as well options for the optimization technique (nonlinear least squares/simplex, maximum number of iterations, and tolerance). With the selected solver and optimization method, the unknown independent, dependent, and/or initial state parameters in the model are determined within set ranges. For this study, nonlinear least squares regression was used with Matlab ode45, which is a Rimge-Kutta [3, 4] formula for non-stiff systems. The steps of nonlinear least squares regression are as follows ... 

This works in the same way as the extended method, except that the only operation is the reflection R. Thus the simplex remains regular (in terms of the coded variables) and the same size, throughout the optimization. Special rules apply when R is worse than W, and for stopping the simplex. See references (13) and (16) for the general rules and reference (17) for a pharmaceutical example, where it is used by Mayne, again to optimize a tablet formulation. 

The above example demonstrates a number of changes in size and shape of the domain. With only 3 variable components this was easily done graphically, but where there are more components, systematic methods are necessary, both for selecting components and defining their limits. Screening, or axial designs, described previously for the simplex domain, provide a possible means. Here we extend the treatment of section III.B of chapter 9 to the more usual case of a constrained mixture. 

The truly remarkable thing about the interior point method is that the number of iterations (Newton steps) is almost independent of problem size. For all models solved to date, the number of iterations has been less than 100, and is usually between 20 and 40. (Note, however, that one Newton step involves much more computation than one simplex step.) There are theoretictil and empirical reasons to believe that the number of iterations increases with the log of the number of variables, log( ). Indeed, Marsten et al. (1990) report a family of problems with from 35,000 to 2,000,000 variables for which a regression of iterations vs. log(n) gave an = 0.979. 

The most common of these exact cases are optimization problems that can be modeled as singlecommodity network flows (see Chapter 99). Equivalently, these are the (ILP)s that can be written so that for each variable Xj, at most one constraint coefficient A j equals 1, at most one A j equals —1, and all other equal 0. Such ILP) s are totally unimodular in that any submatrix formed by the Ajj associated with a collection of rows i and a like-sized collection of variables y has determinant 0, 1 or — 1. This is enough to ensure optimal basic solutions to (/LP) (produced, for example, by the simplex algorithm for linear programming) are integer whenever right-hand-side coefficients are all integer. 

The Simplex method represents a more efficient approach using only function values for constructing an irregular polyhedron in parameters space, and moving this polyhedron towards the minimum, while allowing the size to contract or expands to improve the convergence. It is better than the simple-minded one-variable-at-a-time approach, but becomes too slow for many-dimensional functions. 

The resulting set of differential equations was solved numerically by means of an improved Euler method with variable step size control [9]. Parameter optimization was accomplished by a Mead Nelder simplex algorithm [10]. 

The effect of these variable step sizes is that (when two factors are being studied) the triangles making up each simplex are not necessarily equilateral ones. The benefit of the variable step sizes is that initially the simplex is large, and gives rapid... 

---