Sequential Optimization Simplex Method

The relationship between output variables, called the response, and the input variables is called the response function and is associated with a response surface. When the precise mathematical model of the response surface is not known, it is still possible to use sequential procedures to optimize the system. One of the most popular algorithms for this purpose is the simplex method and its many variations (63,64). 

There are two basic types of unconstrained optimization algorithms (I) those reqmring function derivatives and (2) those that do not. The nonderivative methods are of interest in optimization applications because these methods can be readily adapted to the case in which experiments are carried out directly on the process. In such cases, an ac tual process measurement (such as yield) can be the objec tive function, and no mathematical model for the process is required. Methods that do not reqmre derivatives are called direc t methods and include sequential simplex (Nelder-Meade) and Powell s method. The sequential simplex method is quite satisfac tory for optimization with two or three independent variables, is simple to understand, and is fairly easy to execute. Powell s method is more efficient than the simplex method and is based on the concept of conjugate search directions. 

Bindschaedler and Gurny [12] published an adaptation of the simplex technique to a TI-59 calculator and applied it successfully to a direct compression tablet of acetaminophen (paracetamol). Janeczek [13] applied the approach to a liquid system (a pharmaceutical solution) and was able to optimize physical stability. In a later article, again related to analytical techniques, Deming points out that when complete knowledge of the response is not initially available, the simplex method is probably the most appropriate type [14]. Although not presented here, there are sets of rules for the selection of the sequential vertices in the procedure, and the reader planning to carry out this type of procedure should consult appropriate references. 

Because this proceeding is relatively expensive, an effective semi-quantitative method is widely used in optimization, the sequential simplex optimization. Simplex optimization is done without estimation of gradients and setting step widths. Instead of this, the progress of the optimization... 

For the optimization of, for instance, a tablet formulation, two strategies are available a sequential or a simultaneous approach. The sequential approach consists of a series of measurements where each new measurement is performed after the response of the previous one is knovm. The new experiment is planned according to a direction in the search space that looks promising with respect to the quality criterion which has to be optimized. Such a strategy is also called a hill-climbing method. The Simplex method is a well known example of such a strategy. Textbooks are available that describe the Simplex methods [20]. 

For readers with no prior knowledge of optimization methods In the textbook of Box et.al. [14] the basic principles of optimization are also explained. The sequential simplex method is presented in Walters et.al. [20]. Multi-criteria optimization is presented in Chapter 4 on an introductory level. For those readers who want to know more about multicriteria optimization, see the references given in Section 1.3.4 and Chapter 4. 

Simplex Optimization. The sequential simplex method is an example of a sequential multivariate optimization procedure that uses a geometrical figure called a simplex to move through a user-specified of experimental conditions in search of the optimum. Various forms of the simplex have been successfully used in different modes of chromatography, particularly HPLC (40-42) and GC (43-46). 

Given the mere handful of reports in the published literature (6,38,39,52), there are many avenues open in the development of systematic approaches to optimization in SFC. In addition to the opportunities mentioned in the sections on the simplex method and window diagram approach, others include the exploration of other sequential or simultaneous optimization strategies such as optiplex, simulated annealing, method of steepest ascent, etc. that are potentially useful in SFC. 

Mayur et al. (1970) formulated a two level dynamic optimisation problem to obtain optimal amount and composition of the off-cut recycle for the quasi-steady state operation which would minimise the overall distillation time for the whole cycle. For a particular choice of the amount of off-cut and its composition (Rl, xRI) (Figure 8.1) they obtained a solution for the two distillation tasks which minimises the distillation time of the individual tasks by selecting an optimal reflux policy. The optimum reflux ratio policy is described by a function rft) during Task 1 when a mixed charge (BC, xBC) is separated into a distillate (Dl, x DI) and a residue (Bl, xBi), followed by a function r2(t) during Task 2, when the residue is separated into an off-cut (Rl, xR2) and a bottom product (B2, x B2)- Both r2(t)and r2(t) are chosen to minimise the time for the respective task. However, these conditions are not sufficient to completely define the operation, because Rl and xRI can take many feasible values. Therefore the authors used a sequential simplex method to obtain the optimal values of Rl and xR which minimise the overall distillation time. The authors showed for one example that the inclusion of a recycled off-cut reduced the batch time by 5% compared to the minimum time for a distillation without recycled off-cut. 

Section 5.3 describes sequential methods of optimization, in particular the Simplex method. In sequential methods the optimization procedure starts with some initial experiments, inspects the data and defines the location of a new data point which is expected to yield an improved chromatogram. The idea is to approach the optimum step by step in this way. 

In contrast to the simultaneous optimization procedures described in the previous section, the Simplex method is a sequential one. A minimum number of initial experiments is performed, and based on the outcome of these a decision is made on the location of a subsequent data point. This simplest form of a sequential optimization scheme can be characterized by the path 1012 in figure 5.4. 

The Simplex method (and related sequential search techniques) suffers mainly from the fact that a local optimum will be found. This will especially be the case if complex samples are considered. Simplex methods require a large number of experiments (say 25). If the global optimum needs to be found, then the procedure needs to be repeated a number of times, and the total number of experiments increases proportionally. A local optimum resulting from a Simplex optimization procedure may be entirely unacceptable, because only a poor impression of the response surface is obtained. 

Simplex optimization of the primary (program) parameters in programmed temperature GC analysis has been demonstrated [612]. A systematic sequential search [613] may be used as an alternative. The Simplex method may be used to optimize a limited number of program parameters, whereas the latter approach was developed for the optimization of multisegment gradients. The use of interpretive methods has so far only been suggested [614,615]. 

Unlike the other optimization methods described here, the sequential simplex method for optimization neither assumes nor determines a mathematical model for the phenomena studied. 

On the other hand, in situations where the experimental region containing the optimum is not a priori known, a sequential optimization method, for example, a simplex approach, can be applied. Then, the following steps are considered ... 

Different sequential optimization methods can be distinguished, of which the simplex approaches are most commonly applied. They can be further... 

As noted in the introduction, energy-only methods are generally much less efficient than gradient-based techniques. The simplex method [9] (not identical with the similarly named method used in linear programming) was used quite widely before the introduction of analytical energy gradients. The intuitively most obvious method is a sequential optimization of the variables (sequential univariate search). As the optimization of one variable affects the minimum of the others, the whole cycle has to be repeated after all variables have been optimized. A one-dimensional minimization is usually carried out by finding the... 

The simplex methods, as the very name implies, are based on very simple algorithms that can be very easily implemented on analjrtic instruments, transforming the optimization of their performance into an automatic procedure. On the other hand, simplex optimization is always sequential since we can only go to the next step after we know the result of the immediately preceding step. Whereas when we are determining a response siuface we can perform several experiments at the same time to complete a factorial design, the simplex methods only permit us to do one experiment at a time (that is why they are called sequential). This characteristic makes simplex use most convenient for rapid response instruments that are ofl en encountered in anal3d ical chemistry laboratories. 

Efficient experimentation is based on the methods of experimental design and its quantitative evaluation. The latter can be performed by means of mathematical models or graphical representations. Alternatively, sequential methods are apphed, such as the simplex method, instead of these simultaneous methods of experimental optimization. There, the optimum conditions are found by systematic search for the objective criterion, for example, the maximum yield of a chemical reaction, in the space of all experimental variables. 

The most common sequential optimization method is based on the simplex method by Nelder and Mead. A simplex is a geometric figure having a number of vertices equal to one more than the number of factors. A simplex in one dimension is therefore a line, in two dimensions a triangle, in three dimensions a tetrahedron, and in multiple dimensions a hypertetrahedron. 

The best known sequential method for optimization is the simplex method. This requires the experimenter to perform a series of experiments until he or she reaches an optimum. The response surface is not mathematically modeled, but the experimenter follows a series of rules until he or she cannot improve. A simplex is the simplest possible object in N dimensional space, e.g., a line in one dimension, and a triangle in two dimensions. Simplex optimization means that a series of experiments are performed on the corners of such a figure. 

---