Univariate search method

The complexity of the response surface is what makes the optimization of chromatographic selectivity stand out as a particular optimization problem rather than as an example to which known optimization strategies from other fields can be readily applied. This is illustrated by the application of univariate optimization. In univariate optimization (or univariate search) methods the parameters of interest are optimized sequentially. An optimum is located by varying a given parameter and keeping all other parameters constant. In this way, an optimum value is obtained for that particular parameter. From this moment on the optimum value is assigned to the parameter and another parameter is varied in order to establish its optimum value . 

In the actual pattern search method the exploratory moves are made in a way very similar to the univariate search method however, instead of minimizing along the line, we proceed as follows ... 

It is perhaps useful to think of the pattern search method as an attempt to combine the certainty of the multivariate grid method with the ease of the univariate search method, in the sense that it seeks to avoid the enormous numbers of function evaluations inherent in the grid method, without getting involved in the (possibly fruitless and misleading) process of optimizing the variables separately. 

In each step of the univariate search method, p — 1 parameters of the previous estimate vector remain fixed, while a minimum of the objective function is searched for by varying only the remaining parameter over a predetermined range. Starting from the initial parameter vector bo, in p steps all parameters are optimized in a fixed order, after which this process is repeated until each additional decrease of the objective function becomes negligible. 

A more sophisticated version of the sequential univariate search, the Fletcher-Powell, is actually a derivative method where elements of the gradient vector g and the Hessian matrix H are estimated numerically. 

More elaborate techniques have been published in the literature to obtain optimal or near optimal stepping parameter values. Essentially one performs a univariate search to determine the minimum value of the objective function along the chosen direction (Ak ) by the Gauss-Newton method. 

Boas, A. H. (1963b) Chem. Eng., NY 70 (Feb. 4th) 105. How search methods locate optimum in univariate problems. 

Random Search / 6.1.2 Grid Search / 6.1.3 Univariate Search / 6.1.4 Simplex Search Method / 6.1.5 Conjugate Search Directions / 6.1.6 Summary... 

Univariate Search. A variant on the multivariate grid search is the univariate search, sometimes called cyclic search, which again has had a long history in the context of nuclear position and orbital exponent variation. This method is based on the idea that the individual variables refer to co-ordinate axes ei= [1, 0, 0. ., 0]T etc., in the n space, and we can thus perform successive one-dimensional searches along each of the axes. The algorithm is ... 

Figure 1.15. Search methods, (a) Univariate search, (b) Steepest descent.

If xi and X2 are varied one at a time, then the method is known as a univariate search and is the same as carrying out successive line searches. If the step length is determined so as to find the minimum with respect to the variable searched, then the calculation steps toward the optimum, as shown in Figure 1.15a. This method is simple to implement, but can be very slow to converge. Other direct methods include pattern searches such as the factorial designs used in statistical design of experiments (see, for example, Montgomery, 2001), the EVOP method (Box, 1957) and the sequential simplex method (Spendley et ah, 1962). 

As noted in the introduction, energy-only methods are generally much less efficient than gradient-based techniques. The simplex method [9] (not identical with the similarly named method used in linear programming) was used quite widely before the introduction of analytical energy gradients. The intuitively most obvious method is a sequential optimization of the variables (sequential univariate search). As the optimization of one variable affects the minimum of the others, the whole cycle has to be repeated after all variables have been optimized. A one-dimensional minimization is usually carried out by finding the... 

The sequential univariate search or axial iteration method changes one... 

In contrast with algorithms using univariate search, the Rosenbrock method is a so-called acceleration method, which makes the direction and/or the distance (in this case both) of the parameter jumps dependent on the degree of success of the previous parameter jumps. With p parameters, the algorithm proceeds as follows ... 

The advantage of the GA variable selection approach over the univariate approach discussed earlier is that it is a true search for an optimal multivariate regression solution. One disadvantage of the GA method is that one must enter several parameters before it... 

Derivative Methods.—The most well developed of the derivative methods are univariate in nature, that is, they approach the minimum of the multivariate function along a sequence of lines (directions) in the many-dimensional space, and the problem is then to determine an algorithm for the choice of these directions. Usually (but not always) it is required that the current direction be followed until a minimum of the function in that direction is found. One may say that these methods are based on a sequence of onedimensional searches. 

There are some modem methods (such as the memory13 and supermemory14 gradient methods) which are not univariate in nature but which approach the minimum in a sequence of many-dimensional searches. So far, however, such methods have found no use in quantum chemistry and we shall not discuss them further. 

Binary System. Owing to the fact that a binary system which is in SLV equilibrium is univariant, one can not specify the system pressure when the temperature has been fixed already, and vice versa. The pressure (or temperature) has to be determined along with the composition of each phase in an iterative manner. The scheme used to search for the phase compositions in a binary system is essentially the same as that used in the multicomponent system except that the pressure value is modified at each iteration until Equation 1 is satisfied. The pressure is corrected according to the following equation which is based on the one-dimensional Newton s method using the ratio of the fugacity values as the objective function ... 

There are several direct search techniques for minimizing a function of one variable. The methods generally start from an initial estimate and sequentially move toward the minimum. Univariate or line search techniques play a major role in solving subproblems in more complex direct search algorithms. 

The simplest of the methods are of type a. The simplest of these are the so-called axial iteration or univariant techniques. A set of internal coordinates are chosen, and the potential energy is minimized with respect to each coordinate in turn. After completing the 3N - 6 = m independent searches, one returns and repeats the procedure until the change in coordinates is below a given threshold. 

Fig. 9.2 illustrates the Rosenbrock method for a model with two parameters. Because the algorithm attempts searching along the axis of a valley of the objective function S, many evaluations of S can be omitted compared with, for example, the algorithm with univariate... 

---