Optimization techniques Simplex method

The techniques most widely used for optimization may be divided into two general categories one in which experimentation continues as the optimization study proceeds, and another in which the experimentation is completed before the optimization takes place. The first type is represented by evolutionary operations and the simplex method, and the second by the more classic mathematical and search methods. (Each of these is discussed in Sec. V.)... 

Bindschaedler and Gurny [12] published an adaptation of the simplex technique to a TI-59 calculator and applied it successfully to a direct compression tablet of acetaminophen (paracetamol). Janeczek [13] applied the approach to a liquid system (a pharmaceutical solution) and was able to optimize physical stability. In a later article, again related to analytical techniques, Deming points out that when complete knowledge of the response is not initially available, the simplex method is probably the most appropriate type [14]. Although not presented here, there are sets of rules for the selection of the sequential vertices in the procedure, and the reader planning to carry out this type of procedure should consult appropriate references. 

Optimization techniques may be classified as parametric statistical methods and nonparametric search methods. Parametric statistical methods, usually employed for optimization, are full factorial designs, half factorial designs, simplex designs, and Lagrangian multiple regression analysis [21]. Parametric methods are best suited for formula optimization in the early stages of product development. Constraint analysis, described previously, is used to simplify the testing protocol and the analysis of experimental results. 

The method of simplex design of experiments in mathematical theory of experiments belongs to a group of nongradient optimization techniques in multidimensional factor space. As a difference to gradient methods, this method does not require a mathematical model of researched phenomenon or does not require derivative of a response. 

By far the most popular technique is based on simplex methods. Since its development around 1940 by DANTZIG [1951] the simplex method has been widely used and continually modified. BOX and WILSON [1951] introduced the method in experimental optimization. Currently the modified simplex method by NELDER and MEAD [1965], based on the simplex method of SPENDLEY et al. [1962], is recognized as a standard technique. In analytical chemistry other modifications are known, e.g. the super modified simplex [ROUTH et al., 1977], the controlled weighted centroid , the orthogonal jump weighted centroid [RYAN et al., 1980], and the modified super modified simplex [VAN DERWIEL et al., 1983]. CAVE [1986] dealt with boundary conditions which may, in practice, limit optimization procedures. 

The Simplex method (and related sequential search techniques) suffers mainly from the fact that a local optimum will be found. This will especially be the case if complex samples are considered. Simplex methods require a large number of experiments (say 25). If the global optimum needs to be found, then the procedure needs to be repeated a number of times, and the total number of experiments increases proportionally. A local optimum resulting from a Simplex optimization procedure may be entirely unacceptable, because only a poor impression of the response surface is obtained. 

This is a Basic (Microsoft Quickbasic 4.S) version of the simplex algorithm by Richard W. Daniels, An Introduction to Numerical Methods and Optimization Techniques, North Holland Press,... 

When the criterion optimized is a linear function of the operating variables, the feasibility problem is said to be one in linear programming. Being the simplest possible feasibility problem, it was the first one studied, and the publication in 1951 of G. Dantzig s simplex method for solving linear-programming problems (D2) marked the beginning of contemporary research in optimization theory. Section IV,C is devoted to this important technique. 

In the previous example, the technique used to reduce the artificial variables to zero was in fact Dantzig s simplex method. The linear function optimized was the simple sum of the artificial variables. Any linear function may be optimized in the same manner. The process must start with a basic solution feasible with the constraints, the function to be optimized expressed only in terms of the variables not in the starting basis. From these expressions it is decided what nonbasic variable should be brought into the basis and what basic variable should be forced out. The process is iterated until no further improvement is possible. 

There are a niunber of different experimental design techniques that can be used for medium optimization. Four simple methods that have been used successfully in titer improvement programs are discussed below. These should provide the basis for initial medium-improvement studies that can be carried out in the average laboratory. Other techniques requiring a deeper knowledge of statistics, including simplex optimization, multivariate analysis, and principle-component analysis, have been reviewed (5,6). 

The Nelder-Mead downhill simplex method is the optimization technique incorporated in the software package Matlab as fin ins or fininsearch. 

As noted in the introduction, energy-only methods are generally much less efficient than gradient-based techniques. The simplex method [9] (not identical with the similarly named method used in linear programming) was used quite widely before the introduction of analytical energy gradients. The intuitively most obvious method is a sequential optimization of the variables (sequential univariate search). As the optimization of one variable affects the minimum of the others, the whole cycle has to be repeated after all variables have been optimized. A one-dimensional minimization is usually carried out by finding the... 

The Matlab Simulink Model was designed to represent the model stmctuie and mass balance equations for SSF and is shown in Fig. 6. Shaded boxes represent the reaction rates, which have been lumped into subsystems. To solve the system of ordinary differential equations (ODEs) and to estimate unknown parameters in the reaction rate equations, the inter ce parameter estimation was used. This program allows the user to decide which parameters to estimate and which type of ODE solver and optimization technique to use. The user imports observed data as it relates to the input, output, or state data of the SimuUnk model. With the imported data as reference, the user can select options for the ODE solver (fixed step/variable step, stiff/non-stiff, tolerance, step size) as well options for the optimization technique (nonlinear least squares/simplex, maximum number of iterations, and tolerance). With the selected solver and optimization method, the unknown independent, dependent, and/or initial state parameters in the model are determined within set ranges. For this study, nonlinear least squares regression was used with Matlab ode45, which is a Rimge-Kutta [3, 4] formula for non-stiff systems. The steps of nonlinear least squares regression are as follows ... 

This section introduces the two most common empirical optimization strategies, the simplex method and the Box-Wilson strategy. The emphasis is on the latter, as it has a wider scope of applications. This section presents the basic idea the techniques needed at different steps in following the given strategy are given in the subsequent sections. 

Wienke et al. 27 compared the GA with several standard optimization techniques including simulated annealing, grid search, simplex, pattern search, along with local optimization methods, for several test problems. Their conclusion was that the GA consistently outperformed the other methods as measured by the fraction of runs that found the global optimum. 

Darvas [14] had previously taken an important conceptual leap and applied a mathematical technique for optimization— the simplex optimization method—to the lead optimization of natriuretic sulphonamides. By describing molecules by their Hansch parameters (crand it), a common molecule space could be created to describe the series and this space could be walked by the optimization algorithm (Figure 8.4) using the following steps ... 

Another method for finding the minimum is the so-called downhill simplex method [3, 619]. It requires only a function evaluation and does not use either function derivatives or matrix inversitMi. It may be relatively slow if one is trying to optimize many parameters and a shallow minimum, but it will always find a minimum (at least a local minimum). The problem with this technique is that it does not calculate the parameters standard deviations directly. In such cases, it is advisable, after finding the ntinimum by the simplex method, to use these parameters in the CNLS approximation, which should cmiverge quickly and provide standard deviations of the parameters. 

As expected, it appeared that the required CPU time to solve this case varied strongly. This was only to minor part due to differences in time required per reactor simulation, which in particularly depends on the requested accuracy of the integration and the method of integration. Most packages use the backward final differences method for the integration which is a robust but not very fast technique. The differences in CPU time were mainly caused by the different optimization routines used. It appears that so-called indirect search routines, i.e. routines that use the derivatives of the objective function with respect to the parameters to be estimated, are much faster and sufficient robust in comparison with the direct search routines such as the Simplex method [4],... 

The basic simplex optimization method, first described by Spendley and co-workers in 1962 [ 11 ], is a sequential search technique that is based on the principle of stepwise movement toward the set goal with simultaneous change of several variables. Nelder and Mead [12] presented their modified simplex method, introducing the concepts of contraction and expansion, resulting in a variable size simplex which is more convenient for chromatography optimization. 

A number of direct ways for linking atomistic and meso-scale melt simulations have been proposed more recently. The idea behind these direct methods is to reproduce structure or thermodynamics of the atomistic simulation on the meso-scale self-consistently. As this approach is an optimization problem, mathematical optimization techniques are applicable. One of the most robust (but not very efficient) multidimensional optimizers is the simplex optimizer, which has the advantage of not needing derivatives, which are difficult to obtain in the simulation. The simplex method was first applied to optimizing atomistic simulation models to experimental data. We can formally write any observable, like, for example, the density p, as a function of the parameters of the simulation model Bj. In Eq. [2], the density is a function of the Lennard-Jones parameters. 

Search for the overall optimum within the available parameter space Factorial, simplex, regression, and brute-force techniques. The classical, the brute-force, and the factorial methods are applicable to the optimization of the experiment. The simplex and various regression methods can be used to optimize both the experiment and fit models to data. 

Multivariate chemometric techniques have subsequently broadened the arsenal of tools that can be applied in QSAR. These include, among others. Multivariate ANOVA [9], Simplex optimization (Section 26.2.2), cluster analysis (Chapter 30) and various factor analytic methods such as principal components analysis (Chapter 31), discriminant analysis (Section 33.2.2) and canonical correlation analysis (Section 35.3). An advantage of multivariate methods is that they can be applied in... 

---