Newton methods for large problems

As we are considering the case where lies outside of the trust region and p lies within it, the minimum along the curve occurs at 1 2 where p s) crosses the trust region 

The dogleg method allows us to identify quickly a point within the trust region that lowers the model cost function at least as much as the Cauchy point. The advantage over the Newton line search procedure is that the full Newton step is not automatically accepted as the search direction, avoiding the problems inherent in its erratic size and direction when 

In the line search and trust-region Newton metiiods, we must obtain the full Newton step by solving a Unear system at each Newton iteration, 

B 1 1 is sparse, this product is fast to compute and we only need to store the nonzero elements of5[ i. 

BFGS can be applied to large problems when the Hessian is sparse if the update formula (5.38) is provided with the sparsity pattern so that only the nonzero positions are stored and updated. Alternatively, only die most recent gradient vectors may be retained and used in (5.38). Both approaches allow the construction of approximate Hessians with limited memory usage. For a more detailed discussion of memory-efficient BFGS methods, consult Nocedal Wright (1999). 

---