The Fletcher-Reeves Algorithm

The most frequently used methods fall between the Newton method and the steepest descents method. These methods avoid direct calculation of the Hessian (the matrix of second derivatives) instead they start with an approximate Hessian and update it at every iteration. 

The conjugate gradient method is one of the oldest in the Retcher-Reeves approach, the search direction is given by 

A more detailed discussion of such methods can be found in specialist books dealing with optimization (Fletcher, 1981 Powell, 1985). 

Suppose we have a wavefunction P, which for the sake of argument I will assume to be real. The expectation value of the energy is 

H now differentiate these expressions with respect to some parameter a that (jould be a Cartesian coordinate, the component of an applied electric field, or whatever. We then have 

The first and third terms on the right-hand side are equal for a real wavefunction. differentiating the normalization condition gives 

In the early days of quantum chemistry, it was argued that terms such qs f (d i /da)H l dr were therefore zero, as substitution of into 

S referred to as the Hellman-Feynman theorem. It was widely used to investi-ate isoelectronic processes such as isomerizations X — Y, barriers to internal Otation, and bond extensions where the only changes in the energy are due to hanges in the positions of the nuclei and so the energy change can be calculated tom one-electron integrals. 

---

Search trajectory for the Fletcher-Reeves algorithm (the numbers designate the iteration). 

For a purely quadratic function the Polak-Ribiere method is identical to the Fletcher-Reeves algorithm as all gradients will be orthogonal. However, most functions of interest, including those used in molecular modelling, are at best only approximately quadratic. Polak and Riviere claimed that their method performed better than the original Fletcher-Reeves algorithm, at least for the functions that they examined. 

Several variants of the conjugate gradients method have been proposed. The formulatior given in Equation (5.7) is the original Fletcher-Reeves algorithm. Polak and Ribiere proposed an alternative form for the scalar constant 7) ... 

EXAMPLE 6.2 APPLICATION OF THE FLETCHER-REEVES CONJUGATE GRADIENT ALGORITHM... 

The conjugate gradient (CG) algorithm is one of the older methods and, strictly speaking, is not a quasi-Newton method. However, it is the method of choice for very large problems where the storage of the Hessian is not practicable. In the Fletcher-Reeves approach the search direction is given by... 

Different formulas for Pk have been developed for the nonlinear case, though they all reduce to the same expressions for convex quadratic functions. These variants exhibit different behavior in practice. Three of the best known algorithms are due to Fletcher-Reeves, Polak-Ribi re, and Hestenes-Stiefel. They are given by ... 

---