Fletcher nonlinear optimization

Figure 16 Root-mean-squared error progression plot for Fletcher nonlinear optimization and back-propagation algorithms during training.

Wolfe Reduced Gradient Method to the Case of Nonlinear Constraints", in R. Fletcher (Ed.)"Optimization, Academic Press, London, 1969, pp49-64. 

BFGS Broyden Fletcher Goldfarb Shanno algorithm, an iterative metho for solving unconstrained nonlinear optimization problems ... 

References General references include the following textbooks. For nonlinear programming, Fletcher, R, Practical Methods of Optimization, Wiley (1987) Nocedal, J., and S. J. Wright, Optimization, Springer, New York (1999) ... 

Abadie, J., and Carpentier, J. (1969), Generalization of the Wolfe Reduced Gradient Method to the Case of Nonlinear Constraints, in Optimization, R. Fletcher, Ed., Academic Press, New York. Armijo, L. (1966), Minimization of Functions having Lipschitz Continuous First-Partial Derivatives, Pacific J. Mathematics, Vol. 16, No. 1, pp. 1-3. 

BFGS = Broyden-Fletcher-Goldfarb-Shanno optimizer NLLSQ = nonlinear least squares optimization PECi = pair excitation Cl NAO-PC = natural atomic orbital-point charge model. 

---