Inverse Hessian matrix

Xk) is the inverse Hessian matrix of second derivatives, which, in the Newton-Raphson method, must therefore be inverted. This cem be computationally demanding for systems u ith many atoms and can also require a significant amount of storage. The Newton-Uaphson method is thus more suited to small molecules (usually less than 100 atoms or so). For a purely quadratic function the Newton-Raphson method finds the rniriimum in one step from any point on the surface, as we will now show for our function f x,y) =x + 2/. 

At each iteration k, the new positions are obtained from the current positions x, the gradient gj. and the current approximation to the inverse Hessian matrix... 

The above formula is obtained by differentiating the quadratic approximation of S(k) with respect to each of the components of k and equating the resulting expression to zero (Edgar and Himmelblau, 1988 Gill et al. 1981 Scales, 1985). It should be noted that in practice there is no need to obtain the inverse of the Hessian matrix because it is better to solve the following linear system of equations (Peressini et al. 1988)... 

These methods utilize only values of the objective function, S(k), and values of the first derivatives of the objective function. Thus, they avoid calculation of the elements of the (pxp) Hessian matrix. The quasi-Newton methods rely on formulas that approximate the Hessian and its inverse. Two algorithms have been developed ... 

For the quasi-Newton method discussed in Section 6.4, give the values of the elements of the approximate to the Hessian (inverse Hessian) matrix for the first two stages of search for the following problems ... 

SHELXL (Sheldrick and Schneider, 1997) is often viewed as a refinement program for high-resolution data only. Although it undoubtedly offers features needed for that resolution regime (optimization of anisotropic temperature factors, occupancy refinement, full matrix least squares to obtain standard deviations from the inverse Hessian matrix, flexible definitions for NCS, easiness to describe partially... 

There are a few points with respect to this procedure that merit discussion. First, there is the Hessian matrix. With elements, where n is the number of coordinates in the molecular geometry vector, it can grow somewhat expensive to construct this matrix at every step even for functions, like those used in most force fields, that have fairly simple analytical expressions for their second derivatives. Moreover, the matrix must be inverted at every step, and matrix inversion formally scales as where n is the dimensionality of the matrix. Thus, for purposes of efficiency (or in cases where analytic second derivatives are simply not available) approximate Hessian matrices are often used in the optimization process - after aU, the truncation of the Taylor expansion renders the Newton-Raphson method intrinsically approximate. As an optimization progresses, second derivatives can be estimated reasonably well from finite differences in the analytic first derivatives over the last few steps. For the first step, however, this is not an option, and one typically either accepts the cost of computing an initial Hessian analytically for the level of theory in use, or one employs a Hessian obtained at a less expensive level of theory, when such levels are available (which is typically not the case for force fields). To speed up slowly convergent optimizations, it is often helpful to compute an analytic Hessian every few steps and replace the approximate one in use up to that point. For really tricky cases (e.g., where the PES is fairly flat in many directions) one is occasionally forced to compute an analytic Hessian for every step. 

Newton s method and quasi-Newton techniques make use of second-order derivative information. Newton s method is computationally expensive because it requires analytical first-and second-order derivative information, as well as matrix inversion. Quasi-Newton methods rely on approximate second-order derivative information (Hessian) or an approximate Hessian inverse. There are a number of variants of these techniques from various researchers most quasi-Newton techniques attempt to find a Hessian matrix that is positive definite and well-conditioned at each iteration. Quasi-Newton methods are recognized as the most powerful unconstrained optimization methods currently available. 

George, et al. (28) implemented Powell s method (29), a quasi-Newton method, that uses approximations to the Hessian matrix and its inverse, H u) and H u l, to calculate new compositions. 

Specific enthalpy Approximation to Hessian matrix Approximation to inverse of Hessian matrix Identity matrix... 

The basic idea in these methods is building up curvature information progressively. At each step of the algorithm, the current approximation to the Hessian (or inverse Hessian, as we shall see) is updated by using new gradient information. The updated matrix itself is not necessarily stored explicitly, as the updating procedure may be defined compactly in terms of a small set of stored vectors. This economizes memory requirements considerably and increases the appeal to large-scale applications. 

The number of line searches, energy and gradient evaluations are given in Table VI for both the Newton and Quasi-Newton methods. Table VI clearly indicates that the use of an exact inverse Hessian requires less points to arrive at the optimum geometry. However in Table VI we have not included the relative computer times required to form the second derivative matrix. If this is taken into account, then the Newton s method with its requirement for an exact Hessian matrix is considerably slower than the quasi-Newton procedures. 

Since q, p ) is the degree of freedom corresponding to the negative eigenvalue of the Hessian matrix of the potential, the sign of its lowest potential term is minus—that is, the inverse harmonic potential. For other degrees of freedom, the lowest-order terms of their potentials describe harmonic oscillators. Therefore, they are vibrational degrees of freedom. In Eq. (16), we normalize the coefficients of the Hamiltonian so that they are written cls ( m = 1,2,..., N). 

Finally, for this section we note that the valence interactions in Eq. [1] are either linear with respect to the force constants or can be made linear. For example, the harmonic approximation for the bond stretch, 0.5 (b - boV, is linear with respect to the force constant If a Morse function is chosen, then it is possible to linearize it by a Taylor expansion, etc. Even the dependence on the reference value bg can be transformed such that the force field has a linear term k, b - bo), where bo is predetermined and fixed, and is the parameter to be determined. The dependence of the energy function on the latter is linear. [After ko has been determined the bilinear form in b - bo) can be rearranged such that bo is modified and the term linear in b - bo) disappears.] Consequently, the fit of the force constants to the ab initio data can be transformed into a linear least-squares problem with respect to these parameters, and such a problem can be solved with one matrix inversion. This is to be distinguished from parameter optimizations with respect to experimental data such as frequencies that are, of course, complicated functions of the whole set of force constants and the molecular geometry. The linearity of the least-squares problem with respect to the ab initio data is a reflection of the point discussed in the previous section, which noted that the ab initio data are related to the functional form of empirical force fields more directly than the experimental data. A related advantage in this respect is that, when fitting the ab initio Hessian matrix and determining in this way the molecular normal modes and frequencies, one does not compare anharmonic and harmonic frequencies, as is usually done with respect to experimental results. 

Quasi-Newton methods attempt to achieve the very fast convergence of Newton s method without having to calculate the Hessian matrix explicitly. The idea is to use gradients to successively build up an approximation to the inverse Hessian. For Newton s method, new directions are taken as... 

In these methods, also known as quasi-Newton methods, the approximate Hessian is improved (updated) based on the results in previous steps. For the exact Hessian and a quadratic surface, the quasi-Newton equation = HAq and its analogue H Ag - = Aq - must hold (where Ag - = g - g and similarly for Aq - ). These equations, which have only n components, are obviously insufficient to determine the n(n + l)/2 independent components of the Hessian or its inverse. Therefore, the updating is arbitrary to a certain extent. It is desirable to have an updating scheme that converges to the exact Hessian for a quadratic function, preserves the quasi-Newton conditions obtained in previous steps, and—for minimization—keeps the Hessian positive definite. Updating can be performed on either F or its inverse, the approximate Hessian. In the former case repeated matrix inversion can be avoided. All updates use dyadic products, usually built... 

There are a number of variations on the Newton-Raphson method, many of which aim to eliminate the need to calculate the full matrix of second derivatives. In addition, a family of methods called the quasi-Newton methods require only first derivatives and gradually construct the inverse Hessian matrix as the calculation proceeds. One simple way in which it may be possible to speed up the Newton-Raphson method is to use the same Hessian matrix for several successive steps of the Newton-Raphson algorithm with only the gradients being recalculated at each iteration. 

Calculation of the inverse Hessian matrix can be a potentially time-consuming operation that represents a significant drawback to the pure second derivative methods such as Newton-Raphson. Moreover, one may not be able to calculate analytical second derivatives, which are preferable. The quasi-Newton methods (also known as variable metric methods) gradually build up the inverse Hessian matrix in successive iterations. That is, a sequence of... 

---