Hessian, in optimization methods

Random Phase Approximation (RPA) method, SINDO modelT84 Synchronous reaction, 356 Updated Hessian, in optimization methods. 

Unconstrained optimization methods [W. II. Press, et. ah, Numerical Recipes The An of Scieniific Compulime.. Cambridge University Press, 1 9H6. Chapter 101 can use values of only the objective function, or of first derivatives of the objective function. second derivatives of the objective function, etc. llyperChem uses first derivative information and, in the Block Diagonal Newton-Raphson case, second derivatives for one atom at a time. TlyperChem does not use optimizers that compute the full set of second derivatives (th e Hessian ) because it is im practical to store the Hessian for mac-romoleciiles with thousands of atoms. A future release may make explicit-Hessian meth oils available for smaller molecules but at this release only methods that store the first derivative information, or the second derivatives of a single atom, are used. 

Numerous optimization methods aim at approximating the Hessian (or its inverse) in various ways. 

Some of the most important variations are the so-called Quasi-Newton Methods, which update the Hessian progressively and therefore economize compute requirements considerably. The most successful scheme for that purpose is the so-called BFGS update. For a detailed overview of the mathematical concepts, see [78, 79] an excellent account of optimization methods in chemistry can be found in [80]. 

Several attempts have been made to devise simpler optimization methods than the lull second order Newton-Raphson approach. Some are approximations of the full method, like the unfolded two-step procedure, mentioned in the preceding section. Others avoid the construction of the Hessian in every iteration by means of update procedures. An entirely different strategy is used in the so called Super - Cl method. Here the approach is to reach the optimal MCSCF wave function by annihilating the singly excited configurations (the Brillouin states) in an iterative procedure. This method will be described below and its relation to the Newton-Raphson method will be illuminated. The method will first be described in the unfolded two-step form. The extension to a folded one-step procedure will be indicated, but not carried out in detail. We therefore assume that every MCSCF iteration starts by solving the secular problem (4 39) with the consequence that the MC reference state does not... 

Only the gradient vector is calculated exactly in approximate optimization methods like the super-CI approach. This information about the exact gradients can be used to improve the convergence of the calculation via a procedure, that updates the approximate Hessian, which is implicitly used in the calculation. Suppose that we know the gradient at two consecutive points in a sequence of iterations, p(n+1) and p(n). Let us expand the gradient around the point p(n+1) ... 

However, in a quantum chemical context there is often one overwhelming difficulty that is common to both Newton-like and variable-metric methods, and that is the difficulty of storing the hessian or an approximation to its inverse. This problem is not so acute if one is using such a method in optimizing orbital exponents or internuclear distances, but in optimizing linear coefficients in LCAO type calculations it can soon become impossible. In modern calculations a basis of say fifty AOs to construct ten occupied molecular spin-orbitals would be considered a modest size, and that would, even in a closed-shell case, give one a hessian of side 500. In a Newton-like method the problem of inverting a matrix of such a size is a considerable... 

Section III then introduces the various approximate energy expressions that are used to determine the wavefunction corrections within each iteration of the MCSCF optimization procedure. Although many of these approximate energy expressions are defined in terms of the same set of intermediate quantities (i.e. the gradient vector and Hessian matrix elements), these expressions have some important formal differences. These formal differences result in MCSCF methods that have qualitatively different convergence characteristics. 

The discussion above has centred around full second-order optimization methods where no further approximations have been made. Computationally such procedures involve two major steps which consume more than 90% of the computer time the transformation of two-electron integrals and the update of the Cl vector. The latter problem was discussed, to some extent, in the previous section. In order to make the former problem apparent, let us write down the explicit formula for one of the elements of the orbital-orbital parts of the Hessian matrix (31), corresponding to the interaction between two... 

J. D. Head and M. C. Zerner, Chem. Phys. Lett., 131,359 (1986). An Approximate Hessian for Molecular Geometry Optimization. (Introduced is an approximate analytical Hessian that decreases the amount of work required by a factor of N in ZDO methods, where N is the size of the basis.)... 

The final factors affecting optimization are the choice for the initial Hessian and the method used to form Hessians at later steps. As discussed in Section 10.3.1, QN methods avoid the costly computation of analytic Hessians by using Hessian updating. In that section, we also showed the mathematical form of some common updating schemes and pointed out that the BEGS update is considered the most appropriate choice for minimizations. What may not have been obvious from Section 10.3.1 is that the initial... 

A brief description of optimizations methods will be given (also see refs. 41-44). In contrast to other fields, in computational chemistry great effort is given to reduce the number of function evaluations since that part of the calculation is so much more time consuming. Since first derivatives are now available for almost all ab initio methods, the discussion will focus on methods where first derivatives are available. The most efficient methods, called variable metric or quasi-Newton methods, require an approximate matrix of second derivatives that can be updated with new information during the course of the optimization. Some of the more common methods have different equations for updating the second derivative matrix (also called the Hessian matrix). 

Emet, 2002). The optimization problem was solved with the ECP method described in Westerlund and Pora (2002). Comparisons were carried out using an implementation of the BB-method for MINLP problems by Leyffer (1999). Whereas the applied BB-method requires both gradient and Hessian information, the ECP-method only requires gradient information. The derivatives needed in each method were thus approximated using finite differences. 

We have seen that the methods for optimization of molecular geometries can be divided into two broad classes second-order methods which require the exact gradient and Hessian in each iteration, and first-order (quasi-Newton) methods which require the gradient only. Both methods are in widespread use, but the first-order methods are more popular since analytical energy gradients are available for almost all electronic structure methods, whereas analytical Hessians are not. Also, the simpler first-order methods usually perform quite well, converging in a reasonable number of iterations in most cases. 

True quasi-Newton schemes attempt to sidestep any calculation of the exact Hessian. In such cases, an approximate Hessian must be estimated by the program or by the user. How should such an estimate be obtained One attractive possibility is to utilize experimental data on vibrational frequencies, or to transfer estimates from one system to a related system. Here is where the choice of coordinate system can be crucial. In general, such estimates are likely to provide information only about, in effect, the normal modes of vibration of a system they specify only the diagonal elements of the Hessian in a particular coordinate system. There are few sources of empirical information about coupling force constants, for example, and few sources about force constants expressed in Cartesian coordinates. Also Hessian information in Cartesian coordinates is rarely transferable from one molecule to another. Hence, for first-order methods the choice of coordinate system in which the optimization is performed is strongly influenced by the need to obtain Hessian information. 

---