Inexact Newton methods

Additional examples of the use of functions in evaluating the gradient and/or Hessian in the BzzMatrixSparseSymmetricLocked class can be found in 

Contrary to the common perception, vhen a problem s dimensions are very large, Newton s methods (somewhat modified) may be more efficient than quasi-Newton methods, which are preferable for small- and medium-scale problems. 

The reason is simple no methods that update the factorization of Hessian (see Section 3.7) can preserve its sparsity without worsening the efficiency of the method itself. 

On the other hand, Newton s method, if modified as discussed in Section 3.6 to force convergence when the prevision is unsatisfactory, can be used if 

As above, a BzzMatrixSparseSymmetricLocked class object can use the function BuildGradientAndHessian to evaluate the function Hessian preserving its sparsity. 

---

Dembo, R. S. S. C. Eisenstat and T. Steihang. Inexact Newton Methods. SIAM J Num Anal 19 40CM08 (1982). 

P. Deuflhard, in Proceedings of the Copper Mountain Conference on Iterative Methods, Copper Mountain, Colorado, April 1—5, 1990. Global Inexact Newton Methods for Very Large Scale Nonlinear Problems. 

These methods are called the inexact Newton methods since the solution of the linear system (4.19) is inexact. 

Chapter 4 has been devoted to large-scale unconstrained optimization problems, where problems related to the management of matrix sparsity and the ordering of rows and columns are broached. Hessian evaluation, Newton and inexact Newton methods are discussed. 

The BFGS formula is generally preferred to (26) since computational results have shown that it requires considerably less effort, especially when inexact line searches are used. Quasi-Newton methods, also referred to as variable metric methods, are much more widely used than either the steepest descent or Newton s method. For additional details and computational comparisons, see Fletcher (1987, pp. 44-74). 

Inexact Newton s methods update only a portion of the Hessian and solve the linear system using an iterative method. This family of methods is a hybrid (classical Newton method and conjugate direction method). These methods are useful in solving very large problems (see Chapter 4). 

---