DFP-update

This BFGS update is only slightly more complicated than the DFP update, and this very slight extra computation is more than compensated for by the extra speed of convergence that is observed in practice. 

It should be stressed that the DFP-update preserves the positive definiteness only if condition (25) is fulfilled. Because this fact is often overlooked, we give some explanations hereto. 

Therefore, we want to confine the subsequent considerations to the updates which are defined by Eq.(33) (Broydens s class of updates). The updates of this class which can be written as a convex linear combination (i.e. [0,1]) of the BFGS- and DFP-update are of... 

Now the question arises Under what conditions are the BFGS- (0=1) and the DFP-update (0=0) the optimally conditioned updates among the positive definite updates of the Broyden s class when p is the (juasi-Newton vector ... 

Hence, the DFP-update may be optimally conditioned only if one of the three conditions... 

The rank two BFGS and DFP formula form two non-unit eigenvalues on the first update, again reflecting the rank two nature of the update. Both formula, as expected, preserve the positive definite nature of the Hessian. 

The BFGS correction formula was discovered independently and more-or-less simultaneously by Broyden (1970), Fletcher (1970), Goldfkrb (1970) and Shanno (1970). The idea is to pretend one is using a DFP-type scheme to estimate F instead of F1, i.e. take F0 = I and try to get Fk pk q The update scheme would be the same as above with p s and q s interchanged. Again, the subscript k is omitted from every term on the right side, and we write ... 

The first formulation of the rank-2 updating was the Davidon-Fletcher-Powell (DFP) formula that updates the inverse Hessian matrix ... 

Although one is the complementary formulation of the other, it is worth noting that DFP and BFGS are not equivalent. In fact, the BFGS update for B produces, by taking the inverse, a corresponding update for H, in the form... 

Note that rtk = 0 yields the DFP method and = 1 the BFGS method. The optimally conditioned (OC) method chooses to minimize the condition number of the Hessian (the condition number is the ratio of the largest to the smallest eigenvalue), thereby improving the behavior of the optimization. The CG, MS and DFP methods are also special cases of the Huang family of algorithms. Equations (7)-(10) can also be used to update the Hessian, B, rather than its inverse, H, provided that Ax and Ag are interchanged when H is replaced by B. ... 

If necessary, we shall use the abbreviations BFGS, DFP, MS etc. as lower indices of a matrix instead of + or c to indicate that this matrix is computed by the corresponding update. 

There is a close relationship between the DFP- and the BFGS-update. If... 

Remark 1 Step 5 must still be specified. The matrix has been introduced to indicate that the matrix k is modified by a (low-rank) correction matrix. When step 5 is reset by a specific update formula, the quasi-Newton method is named after that update (for instance BFGS-method, DFP-method, Broyden-method,. ..). 

---