Numerical search direct methods

Direct search methods use only function evaluations. They search for the minimum of an objective function without calculating derivatives analytically or numerically. Direct methods are based upon heuristic rules which make no a priori assumptions about the objective function. They tend to have much poorer convergence rates than gradient methods when applied to smooth functions. Several authors claim that direct search methods are not as efficient and robust as the indirect or gradient search methods (Bard, 1974 Edgar and Himmelblau, 1988 Scales, 1986). However, in many instances direct search methods have proved to be robust and reliable particularly for systems that exhibit local minima or have complex nonlinear constraints (Wang and Luus, 1978). 

The method of steepest descent uses only first-order derivatives to determine the search direction. Alternatively, Newton s method for single-variable optimization can be adapted to carry out multivariable optimization, taking advantage of both first- and second-order derivatives to obtain better search directions1. However, second-order derivatives must be evaluated, either analytically or numerically, and multimodal functions can make the method unstable. Therefore, while this method is potentially very powerful, it also has some practical difficulties. 

Specifically, in Chapter 3 we create a surface for a transcendental function /(a, y) as an elevation matrix whose zero contour, expressed numerically as a two row matrix table of values, solves the nonlinear CSTR bifurcation problem. In Chapter 6 we investigate multi-tray processes via matrix realizations in Chapter 5 we benefit from the least squares matrix solution to find search directions for the collocation method that helps us solve BVPs and so on. Matrices and vectors are everywhere when we compute numerically. That is, after the laws of physics and chemistry and differential equations have helped us find valid models for the physico-chemical processes. 

The Fink method of numerical searching of the data file relies more on d spacings than on intensities. It was originally designed for use with electron diffraction patterns, where observed line intensities are not always directly related to structure and therefore not always a reliable guide to identification. 

With all the necessary ingredients in place, the task is now to derive a reliable force field. In an automated refinement, the first step is to define in machine-readable form what constitutes a good force field. Following that, the parameters are varied, randomly or systematically (15,42). For each new parameter set, the entire data set is recalculated, to yield the quality of the new force field. The best force field so far is retained and used as the basis for new trial parameter sets. The task is a standard one in nonlinear numerical optimization many efficient procedures exist for selection of the optimum search direction (43). Only one recipe will be covered here, a combination of Newton-Raphson and Simplex methods that has been successfully employed in several recent parameterization efforts (11,19,20,28,44). 

The problem of local minima in function minimization or optimization problems has given rise to the development of a variety of algorithms which are able to seek global minima. Traditional gradient based optimizers proceed by the selection of fruitful search directions and subsequent numerical one-dimensional optimization along these paths. Such methods are therefore inherently prone to the discovery of local minima in the vicinity of their starting point, as illustrated in Fig. 5.4(b). This property is in fact desirable in... 

They (in theory in classical analysis without round-off errors) linearly converge to the minimum. The gradient tends to zero close to the minimum. Methods that exploit the gradient can, therefore, encounter problems in the neighborhood of the minimum since the search direction is numerically inaccurate. 

As discussed in the previous section a brute-force molecular-dynamics simulation of gas-surface dynamics, although simple in principle, is a large computational problem. Though these direct methods will continue to be of use, particularly in providing numerical benchmarks for the calibration of more approximate methods, it will prove useful to search for more efficient methods. The chief defect of direct methods when applied to gas-solid scattering is that the essentially harmonic chatracter of the lattice is not fully exploited. We expect that the strong, direct interaction with the solid will involve a relatively small number of lattice... 

Basically two search procedures for non-linear parameter estimation applications apply. (Nash and Walker-Smith, 1987). The first of these is derived from Newton s gradient method and numerous improvements on this method have been developed. The second method uses direct search techniques, one of which, the Nelder-Mead search algorithm, is derived from a simplex-like approach. Many of these methods are part of important mathematical computer-based program packages (e.g., IMSL, BMDP, MATLAB) or are available through other important mathematical program packages (e.g., IMSL). 

Deterministic methods. Deterministic methods follow a predetermined search pattern and do not involve any guessed or random steps. Deterministic methods can be further classified into direct and indirect search methods. Direct search methods do not require derivatives (gradients) of the function. Indirect methods use derivatives, even though the derivatives might be obtained numerically rather than analytically. 

Banga et al. [in State of the Art in Global Optimization, C. Floudas and P. Pardalos (eds.), Kluwer, Dordrecht, p. 563 (1996)]. All these methods require only objective function values for unconstrained minimization. Associated with these methods are numerous studies on a wide range of process problems. Moreover, many of these methods include heuristics that prevent premature termination (e.g., directional flexibility in the complex search as well as random restarts and direction generation). To illustrate these methods, Fig. 3-58 illustrates the performance of a pattern search method as well as a random search method on an unconstrained problem. 

---