Nonlinear Conjugate Gradient

Extensions of the linear CG method to nonquadratic problems have been developed and extensively researched.66-78 In the several existing variants, the basic idea is to avoid matrix operations altogether and simply express the search directions recursively as 

Three of the best known settings for (3 are titled the Fletcher-Reeves (FR), Polak-Ribiere (PR), and Hestenes-Stiefel (HS) formulas.66-71 77 78 They are given by the formulas 

The last two formulas are generally preferred in practice, though the first has better theoretical global convergence properties. In fact, very recent research has focused on combining these practical and theoretical properties for construction of more efficient schemes.77 78 The simple modification of 

The quality of line search in these nonlinear CG algorithms is crucial. (Typically, line searches are used rather than the trust region methods.) Adjustments must be made not only to preserve the mutual conjugacy of the search directions—a property critical for finite termination of the method—but also to ensure that each generated direction is one of descent. A technique known as 

---

The descent condition is used to define the algorithmic sequence that generates p but is not always tested in practice. In reality, numerical errors can lead to departure from theoretical expectations. Thus, it is often necessary to check explicitly for the descent property of p t from equation (10), especially in nonlinear conjugate gradient methods which are very sensitive to roundoff. 

Two important developments have emerged in modem optimization research in connection with QN methodology. The first is the development of limited-memory versions, in which the inverse Hessian approximation at step k only incorporates curvature information generated at the last few m steps (e.g., m = 5). The second is the emergence of insightful analyses that explain the relationship between QN and nonlinear conjugate gradient methods. 

Nonlinear conjugate gradient and various Newton methods are quite popular, but algorithmic details and parameters vary... 

L. Adams and J. L. Nazareth (eds.), Linear and Nonlinear Conjugate Gradient-Related Methods , SIAM, Philadelphia, PA, 1996. 

The need in Newton s method to store and to solve a linear system (5.50) at each iteration poses a sigitificant challenge for large optimization problems. For large problems with Hessians tiiat are dense or whose sparsity padems are unknown, the tricks above cannot be used. Instead, the nonlinear conjugate gradient method, which does not require any curvature knowledge, is recommended. 

We find a local minimum by applying either the nonlinear conjugate gradient method or, as below, a variation of Newton s method. For the latter technique, we use the Hessian... 

SCF will soon be the O(N ) operations (e.g., diagonalization). This realization prompted investigation of both the performance of parallel eigensolvers for large processor counts and the adoption of alternative approaches to the SCF nonlinear optimization problem (conjugate gradient and full-second-order approaches),... 

Nonlinear least-squares inversion by the conjugate gradient method This method uses the same ideas as the regularized conjugate gradient method ... 

Using equations (5.102), (5.103), (5.105), and (5.106), we can obtain m iteratively. 5.3.6 The numerical scheme of the regularized conjugate gradient method for nonlinear least-squares vnversion... 

The system of N(M + 1) equations is then solved by using standard nonlinear solvers such as the Newton-Raphson method or the conjugate-gradient minimization algorithm, both of which are described in Press et al. (1992). 

Unconstrained optimization (nonlinear programming), 2546-2553 classictil methods, 2546-2547 conjugate gradient methods, 2552-2553 golden section method, 2547-2549 line search techniques for, 2547 multidimensional search techniques for, 2549-2552... 

Domain mapping of the WMBVP follows the theory by Joseph. The physical fluid domain shown in Fig. 2.14 for the fully nonlinear WMBVP is mapped to a fixed computational fluid domain, and the discretized coupled free-surface boundary conditions are computed by an implicit Crank Nicholson (C N) method. s each iteration of the C-N method, the potential field is computed by the conjugate gradient method. The wavemaker motion E y/h,t) is assumed to be periodic with period T — 2tt/u , and the WMBVP with the surface tension f is given by... 

Multivariable RTO of nonlinear objective functions using function derivatives is recommended with more than two variables. In particular, the conjugate gradient... 

The Estimates, Derivatives and Search parameters can be changed to optimize the solution process. The Search parameter specifies which gradient search method to use the Newton method requires more memory but fewer iterations, the Conjugate method requires less memory but more iterations. The Derivatives parameter specifies how the gradients for the search are calculated the Central derivatives method requires more calculations but may be helpful if the Solver reports that it is unable to find a solution. The Estimates parameter determines the method by which new estimates of the coefficients are obtained from previous values the Quadratic method may improve results if the system is highly nonlinear. 

---