Scaled conjugate gradient

Millam, J. M., Scuseria, G. E., 1997, Linear Scaling Conjugate Gradient Density Matrix Search as an Alternative to Diagonalization for First Principles Electronic Structure Calculations , J. Chem. Phys., 106, 5569. 

Neural network architectures and learning algorithms (see Table 9.1) GA = Genetic Algorithm SCG = Scaled Conjugate Gradient... 

Mailer, M. (1993). A scaled conjugate gradient algorithm for fast supervised learning. Neural Networks 6,525-33. 

Table 1. Confusion matrix of classification results based on Scaled Conjugate Gradient (SCG) Algorithm.

M.F. Moller, A Scaled Conjugate Gradient Algorithm for Fast Supervised Learning, Neural networks, vol. 6, no. 4, pp. 525-533, 1993. 

The Scaled Conjugate Gradient (SCG) algorithm was chosen as the training algorithm. Table 1 shows the MLP parameter settings used. 

The advantage of a conjugate gradient minimizer is that it uses the minimization history to calculate the search direction, and converges faster than the steepest descent technique. It also contains a scaling factor, b, for determining step size. This makes the step sizes optimal when compared to the steepest descent technique. 

It is noted that the Rosenbrock function given by the next equation has been used to test the performance of various algorithms including modified Newton s and conjugate gradient methods (Scales, 1986)... 

The basic difficulty with the steepest descent method is that it is too sensitive to the scaling of/(x), so that convergence is very slow and what amounts to oscillation in the x space can easily occur. For these reasons steepest descent or ascent is not a very effective optimization technique. Fortunately, conjugate gradient methods are much faster and more accurate. 

For additional details concerning the application of conjugate gradient methods, especially to large-scale and sparse problems, refer to Fletcher (1980), Gill et al. (1981), Dembo et al. (1982), and Nash and Sofer (1996). 

---