Steepest Descent optimization method

The SCF wavefunctions from which the electron density is fitted was calculated by means of the GAUSSIAN-90 system of programs [18]. The program QMOLSIM [3] used in the computation of the MQSM allows optimization of the mutual orientation of the two systems studied in order to maximize their similarity by the common steepest-descent, Newton and quasi-Newton algorithms [19]. The DIIS procedure [20] has been also implemented for the steepest-descent optimizations in order to improve the performance of this method. The MQSM used in the optimization procedure are obtained from fitted densities. This speeds the process. The exact MQSM were obtained from the molecular orientation obtained in this optimization procedure. 

Both the simplex and steepest descent optimizations began at 70 °C and 180 s. From there, the simplex method moved toward conditions at higher temperature and lower residence time. Upon reaching the minimum residence time constraint, the simplex contracted, ultimately becoming a one-dimensional search until it reached the optimiun of 99 °C and 30 s near the corner of the design space after 30 experiments. The SNOBFIT algorithm initially performed a randomized, space-fining... 

After these initial preparations, the first step wUl be an optimization of the protein structure. We may initially freeze the protein and allow for some steps of minimization of solvated water molecules (for example, 500 steps of the Steepest Descents (SD) method). 

Aq becomes asymptotically a g/ g, i.e., the steepest descent fomuila with a step length 1/a. The augmented Hessian method is closely related to eigenvector (mode) following, discussed in section B3.5.5.2. The main difference between rational fiinction and tmst radius optimizations is that, in the latter, the level shift is applied only if the calculated step exceeds a threshold, while in the fonuer it is imposed smoothly and is automatically reduced to zero as convergence is approached. 

Example Com pare the steps of a conjugate gradien t min im i/ation with the steepest descent method.. A molecular system can reach a potential minimum after the second step if the first step proceeds from, A to B. If the first step is too large, placing the system at D, the second step still places the system near the tninimum(K ) because the optim i/,er remembers the penultimate step. 

Steepest Descent is the simplest method of optimization. The direction of steepest descen t. g, is just th e n egative of the gradien t vector ... 

The steepest descent by steps method may provide a reasonably good method to begin an optimization when the starting point is far from the minimum. However, it converges slowly near the minimum and it is principally recommended only to initiate optimization when the starting point is particularly bad. 

The steepest descent method is quite old and utilizes the intuitive concept of moving in the direction where the objective function changes the most. However, it is clearly not as efficient as the other three. Conjugate gradient utilizes only first-derivative information, as does steepest descent, but generates improved search directions. Newton s method requires second derivative information but is veiy efficient, while quasi-Newton retains most of the benefits of Newton s method but utilizes only first derivative information. All of these techniques are also used with constrained optimization. 

The gradient can be used to optimize the weight vector according to the method of steepest-descent ... 

Ok) function is sought by repeatedly determining the direction of steepest descent (maximum change in for any change in the coefficients a,), and taking a step to establish a new vertex. A numerical example is found in Table 1.26. An example of how the simplex method is used in optimization work is given in Ref. 143. 

The method of steepest descent uses only first-order derivatives to determine the search direction. Alternatively, Newton s method for single-variable optimization can be adapted to carry out multivariable optimization, taking advantage of both first- and second-order derivatives to obtain better search directions1. However, second-order derivatives must be evaluated, either analytically or numerically, and multimodal functions can make the method unstable. Therefore, while this method is potentially very powerful, it also has some practical difficulties. 

As explained above, the QM/MM-FE method requires the calculation of the MEP. The MEP for a potential energy surface is the steepest descent path that connects a first order saddle point (transition state) with two minima (reactant and product). Several methods have been recently adapted by our lab to calculate MEPs in enzymes. These methods include coordinate driving (CD) [13,19], nudged elastic band (NEB) [20-25], a second order parallel path optimizer method [25, 26], a procedure that combines these last two methods in order to improve computational efficiency [27],... 

An extension of the linearization technique discussed above may be used as a basis for design optimization. Such an application to natural gas pipeline systems was reported by Flanigan (F4) using the so-called constrained derivatives (W4) and the method of steepest descent. We offer a more concise derivation of this method following a development by Bryson and Ho (B14). 

Steepest descent can terminate at any type of stationary point, that is, at any point where the elements of the gradient of /(x) are zero. Thus you must ascertain if the presumed minimum is indeed a local minimum (i.e., a solution) or a saddle point. If it is a saddle point, it is necessary to employ a nongradient method to move away from the point, after which the minimization may continue as before. The stationary point may be tested by examining the Hessian matrix of the objective function as described in Chapter 4. If the Hessian matrix is not positive-definite, the stationary point is a saddle point. Perturbation from the stationary point followed by optimization should lead to a local minimum x. ... 

The basic difficulty with the steepest descent method is that it is too sensitive to the scaling of/(x), so that convergence is very slow and what amounts to oscillation in the x space can easily occur. For these reasons steepest descent or ascent is not a very effective optimization technique. Fortunately, conjugate gradient methods are much faster and more accurate. 

Why is the steepest descent method not widely used in unconstrained optimization codes ... 

In our two-dimensional space, these two search directions are perpendicular to one another. Saying this in more general mathematical terms, the two search directions are orthogonal. This is not a coincidence that occurs just for the specific example we have defined it is a general property of steepest descent methods provided that the line search problem defined by Eq. (3.18) is solved optimally. 

---