Hessian updating

Nocedal, Jorge, and Overton, Michael L., Projected Hessian updating algorithms for non-linearly constrained optimization, SIAM J. Numer. Anal. 22(5), 821 (1985). 

If the exact Hessian is unavailable or computationally expensive, we may use an approximation. Approximate Hessians are usually obtained by one of several Hessian update methods. The update techniques are designed to determine an approximate Hessian B+ at... 

Based on the finite-difference formula Eq. (3.31) all Hessian updates Eire required to fulfill the quasi-Newton condition... 

There Eire other Hessian updates but for minimizations the BFGS update is the most successful. Hessism update techniques are usually combined with line search vide infra) and the resulting minimization algorithms are called quasi-Newton methods. In saddle point optimizations we must allow the approximate Hessian to become indefinite and the PSB update is therefore more appropriate. 

Line searches are often used in connection with Hessian update formulas and provide a relatively stable and efficient method for minimizations. However, line searches are not always successful. For example, if the Hessian is indefinite there is no natural way to choose the descent direction. We may then have to revert to steepest descent although this step makes no use of the information provided by the Hessian. It may also be... 

The trust region method is usually implemented with the exact Hessian. Updated Hessians may also be used but an approximate Hessian usually does not contain enough information about the function to make the trust region reliable in all directions. The trust region method provides us with the possibility to carry out an unbiased search in all directions at each step. An updated Hessian does not contain the information necessary for such a search. 

In this paper we have reviewed and studied several Hessian update procedures for optimizing the geometries of polyatomic molecules. Of the several methods we have studied, many of which we have reported on here in detail, the most reliable and efficient is the BFGS. The weak line search outlined in this work is sufficient to ensure successful optimization even when starting geometries are generated through molecular mechanics [45] or ball and stick models [46]. 

J. Nocedal and M. L. Overton, 1985, Projected Hessian updating algorithms for nonlinearly constrained optimization, SIAM Journal of Numerical Analysis, 22 821-850. 

For systems described by simple empirical potentials containing fewer than 100 atoms we simply calculate analytic first and second derivatives at every step in a transition state search. Because we routinely start these searches from minima, Hessian update techniques do not seem to be competitive. We use the gradients at the present point, n, and the previous point — 1, to estimate the corresponding eigenvalue ... 

The final factors affecting optimization are the choice for the initial Hessian and the method used to form Hessians at later steps. As discussed in Section 10.3.1, QN methods avoid the costly computation of analytic Hessians by using Hessian updating. In that section, we also showed the mathematical form of some common updating schemes and pointed out that the BEGS update is considered the most appropriate choice for minimizations. What may not have been obvious from Section 10.3.1 is that the initial... 

Table 10.3 Comparison of the number of steps required to minimize geometries (QN with RFO algorithm) using a unit matrix, empirically derived Hessian, and analytic Hessian for the initial Hessian followed by Hessian updating and using all analytic Hessians ...

Table 10.7 RMS errors in position (A) for HS reaction path following Hessian updating ...

The idea - on which the quasi-Newton methods are based - is Hessian updating, supposing that the Hessian is available in its approximate form B in x , through the addition of an opportune matrix Q ... 

The main advantage of Hessian updating by means of (3.144) is that it ensures a quadratic termination and thus a finite amount of iterations, even though the onedimensional searches are nonexact. 

More problems arise in this case with respect to Hessian updating in the minimization problem. 

Compare the Hessian evaluate in X to the Hessian updated using the relation (13.18). 

---