Linear CG method

Extensions of the linear CG method to nonquadratic problems have been developed and extensively researched.66-78 In the several existing variants, the basic idea is to avoid matrix operations altogether and simply express the search directions recursively as... 

Nonlinear CG methods form another popular type of optimization scheme for large-scale problems where memory and computational performance are important considerations. These methods were first developed in the 1960s by combining the linear CG method (an iterative technique for solving linear systems Ax = b where A is an /i x /i matrix ) with line-search techniques. The basic idea is that if / were a convex quadratic function, the resulting nonlinear CG method would reduce to solving the Newton equations (equation 27) for the constant and positive-definite Hessian H. 

Still, the linear and nonlinear CG methods play important theoretical roles in the numerical analysis literature as well as practical roles in many numerical techniques see the recent research monograph of Adams and Nazareth for a modem perspective. The linear CG method, in particular, proves ideal for solving the linear subproblem in the truncated Newton method for minimization (discussed next), especially with convergence-accelerating techniques known as preconditioning. 

---