Preconditioned conjugate gradient method

O. Axelsson and G. Lindskog, Numer. Math., 48, 449 (1989). On the Rate of Convergence of the Preconditioned Conjugate Gradient Method. 

Eisenstat, S.C., 1981, Efficient implementation of a class of preconditioned conjugate gradient methods. SIAM J. Sci. Stat. Comput. 2, 1-4... 

Ghosh D, Avery P, Farhat C (2009) FETI-preconditioned conjugate gradient method for large-scale stochastic finite element problems. Int J Numer Methods Eng 80(6-7) 914-931... 

BUSTER/TNT (Bricogne and Irvin, 1996) is another likelihood based refinement package that excels especially in cases in which the model is still severely incomplete (Blanc et al., 2004 Tronrud et al., 1987). It uses atomic parameters but also has a novel solvent and missing model envelope fimc-tion. The optimization method is a preconditioned conjugate gradient as implemented in the TNT package (Tronrud et al., 1987) that had a faithful audience in the pre-likelihood era. 

The recurrence relations for the preconditioned conjugate gradient (PCG) method can be derived from Algorithm [A2] after substituting x = M 1/2x and r + M1/2r. New search vectors d = M 1/2d can be used to derive the iteration process, and then the tilde modifiers dropped. The PCG method becomes the following iterative process. 

Conjugate Gradient (CG) or Preconditioned Conjugate Gradient (PCG) methods (Chapter 6), which can become unstable with nonpositive definite matrices. 

In terms of the computational work per outer Newton step k), TN methods based on preconditioned conjugate gradient require a Hessian-vector product (Hd) at each inner loop iteration, and one solution of a linear system Mz = r where M is the preconditioner. Since M may be sparse, this linear solution often takes a very small percentage of the total CPU time (e.g., <3% ). The benefits of faster convergence generally far outweigh these costs. 

In the work of Lindborg et al [119], the resulting linear equation systems were solved with preconditioned Krylov subspace projection methods [166]. The Poisson equation was solved by a conjugate gradient (CG)-solver, while the other transport equations were solved using a bi-conjugate gradient (BCG)-solver which can handle also non-symmetric equations systems. The solvers were preconditioned with a Jacobi preconditioner. 

More powerful minimization methods can also be employed. The method of conjugate gradients uses the information from earlier steps to choose a more efficient sequence of direaions to minimize along. In addition, preconditioning, which utilizes an easily invertible approximation to A, can enhance convergence. These more advanced techniques require fewer iterations at each grid... 

As the matrices and are symmetric and positive definite in our case, it seems that a good choice of the solution technique is the conjugate gradient (CG) method with a suitable preconditioning. This question will be further discussed in the next section. 

---