Direct search methods

There is a variety of general purpose unconstrained optimization methods that can be used to estimate unknown parameters. These methods are broadly classified into two categories direct search methods and gradient methods (Edgar and Himmelblau, 1988 Gill et al. 1981 Kowalik and Osborne, 1968 Sargent, 1980 Reklaitis, 1983 Scales, 1985). 

Direct search methods use only function evaluations. They search for the minimum of an objective function without calculating derivatives analytically or numerically. Direct methods are based upon heuristic rules which make no a priori assumptions about the objective function. They tend to have much poorer convergence rates than gradient methods when applied to smooth functions. Several authors claim that direct search methods are not as efficient and robust as the indirect or gradient search methods (Bard, 1974 Edgar and Himmelblau, 1988 Scales, 1986). However, in many instances direct search methods have proved to be robust and reliable particularly for systems that exhibit local minima or have complex nonlinear constraints (Wang and Luus, 1978). 

One of the most reliable direct search methods is the LJ optimization procedure (Luus and Jaakola, 1973). This procedure uses random search points and systematic contraction of the search region. The method is easy to program and handles the problem of multiple optima with high reliability (Wang and Luus, 1977, 1978). A important advantage of the method is its ability to handle multiple nonlinear constraints. 

If we have very little information about the parameters, direct search methods, like the LJ optimization technique presented in Chapter 5, present an excellent way to generate very good initial estimates for the Gauss-Newton method. Actually, for algebraic equation models, direct search methods can be used to determine the optimum parameter estimates quite efficiently. However, if estimates of the uncertainty in the parameters are required, use of the Gauss-Newton method is strongly recommended, even if it is only for a couple of iterations. 

Deterministic methods. Deterministic methods follow a predetermined search pattern and do not involve any guessed or random steps. Deterministic methods can be further classified into direct and indirect search methods. Direct search methods do not require derivatives (gradients) of the function. Indirect methods use derivatives, even though the derivatives might be obtained numerically rather than analytically. 

The first example was formulated by Stoecker to illustrate the steepest descent (gradient) direct search method. It is proposed to attach a vapor recondensation refrigeration system to lower the temperature, and consequently vapor pressure, of liquid ammonia stored in a steel pressure vessel, for this would permit thinner vessel walls. The tank cost saving must be traded off against the refrigeration and thermal insulation cost to find the temperature and insulation thickness minimizing the total annual cost. Stoecker showed the total cost to be the sum of insulation cost i = 400jc° 9 (x is the insulation thickness, in.), the vessel cost v = 1000 + 22(p — 14.7)1-2 (p is the absolute pressure, psia), and the recondensation cost r = 144(80 — t)/x (t is the temperature, °F). The pressure is related to the temperature by... 

All the basis functions were given independently variable orbital exponents (CH4, 9 NH3,8 H20,7) and all exponents were optimised by the quadratically convergent direct search method of Fletcher (19). For comparison, the calculations were repeated with the GHOs constrained to have the symmetry of the molecule three independent variables for CH4 (1 sc, sp3, 1 sH) four for NH3 (1 sN, sp3, sp3, 1 sH) and four for H20 (1 sQ, sp3, sp3,1 sH). The most striking qualitative result is the confirmation of the results quoted earlier for H2 when the orbital exponents are all optimised, the GHO basis has the symmetry of the molecule there is no spatial symmetry dilemma.9)... 

Statistical optimization methods other than the Simplex algorithm have only occasionally been used in chromatography. Rafel [513] compared the Simplex method with an extended Hooke-Jeeves direct search method [514] and the Box-Wilson steepest ascent path [515] after an initial 23 full factorial design for the parameters methanol-water composition, temperature and flowrate in RPLC. Although they concluded that the Hooke-Jeeves method was superior for this particular case, the comparison is neither representative, nor conclusive. 

Non-derivative Methods.—Multivariate Grid Search. The oldest of the direct search methods is the multivariate grid search. This has a long history in quantum chemistry as it has been the preferred method in optimizing the energy with respect to nuclear positions and with respect to orbital exponents. The algorithm for the method is very simple. In this and subsequent algorithms we use x to indicate the variables and a to indicate a chosen point. 

Various more-or-less efficient optimization strategies have been developed [46, 47] and can be classified into direct search methods and gradient methods. The direct search methods, like those of Powell [48], of Rosenbrock and Storey [49] and of Nelder and Mead ( Simplex ) [50] start from initial guesses and vary the parameter values individually or combined thereby searching for the direction to the minimum SSR. 

Sometimes it is not necessary to determine a response surface model tor locate the optimum conditions. Hill-climbing by direct search methods, e.g. search along the path of steepest ascent [8] or sequential simplex search [9], will lead to a point on the response surface near the optimum. The computations involved in these methods are rather trivial and do not require a computer and will for this reason not be discussed further in this chapter. Readers who require details of these direct search methods should consult Refs. [1,8,9]. 

Watson, 1968 Rudd, 1968 Masso and Rudd, 1969). Algorithmic methods for selecting the optimal configuration from a given superstructure also began to be developed through the use of direct search methods for continuous variables (Umeda et al, 1972 Ichikawa and Fan, 1973) as well as branch and bound search methods (Lee et al, 1970). 

Optimization can be achieved using various combinations of models and plant data to represent the plant. The correct approach depends on the specific application. Some small-scale optimizations can be addressed using direct search methods, (e.g., Ref . ) These... 

Direct search methods have a disadvantage in that as the number of variables increases, so does the length of time to solve the problem, sometimes leading to exceedingly long search times. They also fail to take into account the information contained by evaluating the derivatives at the point under evaluation. For example if the first derivative is positive, the function is increasing and viceversa. This is useful information when one wants to find the maximum or minimum. 

Use of Direct Search Methods Autocorrelation in Dynamic Systems... 

---