Hessian matrix approximation

Specific enthalpy Approximation to Hessian matrix Approximation to inverse of Hessian matrix Identity matrix... 

At each iteration k, the new positions are obtained from the current positions x, the gradient gj. and the current approximation to the inverse Hessian matrix... 

There are several reasons that Newton-Raphson minimization is rarely used in mac-romolecular studies. First, the highly nonquadratic macromolecular energy surface, which is characterized by a multitude of local minima, is unsuitable for the Newton-Raphson method. In such cases it is inefficient, at times even pathological, in behavior. It is, however, sometimes used to complete the minimization of a structure that was already minimized by another method. In such cases it is assumed that the starting point is close enough to the real minimum to justify the quadratic approximation. Second, the need to recalculate the Hessian matrix at every iteration makes this algorithm computationally expensive. Third, it is necessary to invert the second derivative matrix at every step, a difficult task for large systems. 

The above formula is obtained by differentiating the quadratic approximation of S(k) with respect to each of the components of k and equating the resulting expression to zero (Edgar and Himmelblau, 1988 Gill et al. 1981 Scales, 1985). It should be noted that in practice there is no need to obtain the inverse of the Hessian matrix because it is better to solve the following linear system of equations (Peressini et al. 1988)... 

As seen by comparing Equations 5.6 and 5.12 the steepest-descent method arises from Newton s method if we assume that the Hessian matrix of S(k) is approximated by the identity matrix. 

These methods utilize only values of the objective function, S(k), and values of the first derivatives of the objective function. Thus, they avoid calculation of the elements of the (pxp) Hessian matrix. The quasi-Newton methods rely on formulas that approximate the Hessian and its inverse. Two algorithms have been developed ... 

Difficulty 3 can be ameliorated by using (properly) finite difference approximation as substitutes for derivatives. To overcome difficulty 4, two classes of methods exist to modify the pure Newton s method so that it is guaranteed to converge to a local minimum from an arbitrary starting point. The first of these, called trust region methods, minimize the quadratic approximation, Equation (6.10), within an elliptical region, whose size is adjusted so that the objective improves at each iteration see Section 6.3.2. The second class, line search methods, modifies the pure Newton s method in two ways (1) instead of taking a step size of one, a line search is used and (2) if the Hessian matrix H(x ) is not positive-definite, it is replaced by a positive-definite matrix that is close to H(x ). This is motivated by the easily verified fact that, if H(x ) is positive-definite, the Newton direction... 

For the quasi-Newton method discussed in Section 6.4, give the values of the elements of the approximate to the Hessian (inverse Hessian) matrix for the first two stages of search for the following problems ... 

B approximation to the Hessian matrix used in sequential quadratic pro-... 

JlJ is an approximation for the curvature matrix. It is approximately 0.5 times the Hessian matrix of second derivatives of ssq with respect to the... 

This form is convenient in that the active inequality constraints can now be replaced in the QP by all of the inequalities, with the result that Sa is determined directly from the QP solution. Finally, since second derivatives may often be hard to calculate and a unique solution is desired for the QP problem, the Hessian matrix, is approximated by a positive definite matrix, B, which is constructed by a quasi-Newton formula and requires only first-derivative information. Thus, the Newton-type derivation for (2) leads to a nonlinear programming algorithm based on the successive solution of the following QP subproblem ... 

Gurwitz, C. B., and Overton, M., SQP methods based on approximating a projected Hessian matrix, SIAM J. Sci. Stat. Comput. 10(4) 631 (1989). 

To determine the full set of normal modes in a DFT calculation, the main task is to calculate the elements of the Hessian matrix. Just as we did for CO in one dimension, the second derivatives that appear in the Hessian matrix can be estimated using finite-difference approximations. For example,... 

Because the Hessian matrix is calculated in practice using finite-difference approximations, the eigenvalues corresponding to the translational and rotational modes we have just described are not exactly zero when calculated with DFT. The normal modes for a CO molecule with a finite difference displacements of 8b 0.04 A are listed in Table 5.2. This table lists the calculated... 

Using DFT calculations to predict a phonon density of states is conceptually similar to the process of finding localized normal modes. In these calculations, small displacements of atoms around their equilibrium positions are used to define finite-difference approximations to the Hessian matrix for the system of interest, just as in Eq. (5.3). The mathematics involved in transforming this information into the phonon density of states is well defined, but somewhat more complicated than the results we presented in Section 5.2. Unfortunately, this process is not yet available as a routine option in the most widely available DFT packages (although these calculations are widely... 

No first derivative terms appear here because the transition state is a critical point on the energy surface at the transition state all first derivatives are zero. This harmonic approximation to the energy surface can be analyzed as we did in Chapter 5 in terms of normal modes. This involves calculating the mass-weighted Hessian matrix defined by the second derivatives and finding the N eigenvalues of this matrix. 

The Hessian matrix of the quadratic approximation (3.42) of tine objective... 

There are a few points with respect to this procedure that merit discussion. First, there is the Hessian matrix. With elements, where n is the number of coordinates in the molecular geometry vector, it can grow somewhat expensive to construct this matrix at every step even for functions, like those used in most force fields, that have fairly simple analytical expressions for their second derivatives. Moreover, the matrix must be inverted at every step, and matrix inversion formally scales as where n is the dimensionality of the matrix. Thus, for purposes of efficiency (or in cases where analytic second derivatives are simply not available) approximate Hessian matrices are often used in the optimization process - after aU, the truncation of the Taylor expansion renders the Newton-Raphson method intrinsically approximate. As an optimization progresses, second derivatives can be estimated reasonably well from finite differences in the analytic first derivatives over the last few steps. For the first step, however, this is not an option, and one typically either accepts the cost of computing an initial Hessian analytically for the level of theory in use, or one employs a Hessian obtained at a less expensive level of theory, when such levels are available (which is typically not the case for force fields). To speed up slowly convergent optimizations, it is often helpful to compute an analytic Hessian every few steps and replace the approximate one in use up to that point. For really tricky cases (e.g., where the PES is fairly flat in many directions) one is occasionally forced to compute an analytic Hessian for every step. 

In the case under study, the particular form of the adopted objective functions allows one to overcome this difficulty by introducing a simplified form of the Hessian matrix. In fact, by assuming that the errors emj are small, the second term on the right-hand side of (3.46) can be neglected, and the Hessian matrix can be approximated by the first-order term, which only contains the first derivatives. This assumption can be always done in a neighborhood of the minimum, where these errors tend to the residuals. In conclusion, the form... 

Newton s method and quasi-Newton techniques make use of second-order derivative information. Newton s method is computationally expensive because it requires analytical first-and second-order derivative information, as well as matrix inversion. Quasi-Newton methods rely on approximate second-order derivative information (Hessian) or an approximate Hessian inverse. There are a number of variants of these techniques from various researchers most quasi-Newton techniques attempt to find a Hessian matrix that is positive definite and well-conditioned at each iteration. Quasi-Newton methods are recognized as the most powerful unconstrained optimization methods currently available. 

---