Basic descent

GRG algorithms use a basic descent algorithm described below for unconstrained problems. We state the steps here ... 

In the basic descent algorithm [Al], we iterate until convergence. How exactly do we evaluate the optimality cf an approximate minimum x Furthermore, how do we ensure that computations will not continue unnecessarily (a) when no further progress can be realized or (b) beyond attainable accuracy The accuracy depends on machine precision and cumulative roundoff errors, in addition to algorithmic details. 

Truncated Newton methods were introduced in the early 1980s111-114 and have been gaining popularity ever since.82-109 110 115-123 Their basis is the following simple observation. An exact solution of the Newton equation at every step is unnecessary and computationally wasteful in the framework of a basic descent method. That is, an exact Newton search direction is unwarranted when the objective function is not well approximated by a convex quadratic and/or the initial point is distant from a solution. Any descent direction will suffice in that case. As a solution to the minimization problem is approached, the quadratic approximation may become more accurate, and more effort in solution of the Newton equation may be warranted. 

Algorithm [Al] Basic Descent Using Line Search... 

The line search and the procedure that defines p t form the central components of the basic descent local optimization algorithm above. The work in the line search (number of polynomial interpolations) should be balanced with the overall progress realized in the minimization algorithm. 

Algorithm [A2] Basic Descent Using A Trust Region Subsearch... 

An alternative, and closely related, approach is the augmented Hessian method [25]. The basic idea is to interpolate between the steepest descent method far from the minimum, and the Newton-Raphson method close to the minimum. This is done by adding to the Hessian a constant shift matrix which depends on the magnitude of the gradient. Far from the solution the gradient is large and, consequently, so is the shift d. One... 

Sun J-Q and Ruedenberg K 1993 Quadratic steepest descent on potential energy surfaces. I. Basic formalism and quantitative assessment J. Chem. Phys. 99 5257... 

There are basically two different causes of turbulent eddies. Eddies set in motion by air moving past objects are the result of mechanical turbulence. Parcels of superheated air rising from the heated earth s surface, and the slower descent of a larger portion of the atmosphere surrounding these more rapidly rising parcels, result in thermal turbulence. The size and, hence, the scale of the eddies caused by thermal turbulence are larger than those of the eddies caused by mechanical turbulence. 

The rate of convergence of the Steepest Descent method is first order. The basic difficulty with steepest descent is that the method is too sensitive to the scaling of S(k), so that convergence is very slow and oscillations in the k-space can easily occur. In general a well scaled problem is one in which similar changes in the variables lead to similar changes in the objective function (Kowalik and Osborne, 1968). For these reasons, steepest descent/ascent is not a viable method for the general purpose minimization of nonlinear functions, ft is of interest only for historical and theoretical reasons. 

If the PCA scores are used in subsequent methods as uncorrelated new variables, the optimum number of PCs can be estimated by several techniques. The strategies applied use different criteria and usually give different solutions. Basics are the variances of the PCA scores, for instance, plotted versus the PC number (Figure 3.5, left). According to the definition, the PCI must have the largest variance, and the variances decrease with increasing PC number. For many data sets, the plot shows a steep descent after a few components because most of the variance is covered by the first components. In the example, one may conclude that the data structure is mainly influenced by two driving factors, represented by PCI and PC2. The other... 

The Rearrangement of 1,2,6-Heptatriene. The experimental facts and basic mechanistic idea behind this reaction were outlined in section 1.2. The molecular dynamics study began with a single CASSCF(8,8)/6-31G(d) trajectory started from TSl (see Fig. 21.2) with no kinetic energy (not even ZPE) in any of the real-frequency normal modes. The purpose of such an unphysical trajectory calculation is to see what is the steepest descent path down from the transition state... 

Since separable algebras over k all look basically the same over ks, classifying them is a descent problem of the type we will study more generally in Chapter 17. But since usual Galois theory already classifies separable fields,... 

Now different

basic theorem shows that two forms are isomorphic over R iff there is an isomorphism over S commuting with the descent data. 

The trust radius in the trust region approach is estimated on the basis of the local Hessian s characteristics (positive-definite, positive-semidefinite, indefinite). The basic idea is to choose s nearly in the current negative gradient direction (—gk) when the trust radius is small, and approach the Newton step -Hk x%k as the trust region is increased. (Hk and g denote the Hessian and gradient, respectively, at xk). Note from condition [12] that these two choices correspond to the extremal cases (M = / and M = H) of general descent directions of form p = — M 1g, where M is a positive-definite approximation to the Hessian. 

Many basic algorithms, each with a number of refinements, are useful in the search for a global minimum. Some of these methods are described briefly. These are the grid search, steepest descent, Gauss-Newton, Marquardt, and simplex methods. 

If the WSS surface is seen as similar to a geographical section with hills and valleys (at least for the two-parameter surface), the steepest-descent method would appear to follow the path of a round ball moving towards the minimum. When translated to a computer algorithm, some of the disadvantages become more apparent. The basic approach is to calculate the slope of the surface at the point of the initial (or current) parameter values. This can be calculated as dWSS/ dP over some small increment for each of the parameters P. By combining each of these partial derivatives over all the parameters, the direction of movement towards the minimum can be calculated. The second part of the problem is the distance h to move in thespecified direction. This must be determined by finding the minimum WSS in the direction calculated from the slope. This means extra calculations of the WSS, which makes this process less efficient, especially as it approaches the minimum. The new parameter value is calculated with the help of Eq. (19) ... 

---