Variable metric method

In simple relaxation (the fixed approximate Hessian method), the step does not depend on the iteration history. More sophisticated optimization teclmiques use infonnation gathered during previous steps to improve the estimate of the minunizer, usually by invoking a quadratic model of the energy surface. These methods can be divided into two classes variable metric methods and interpolation methods. 

Owing to the constraints, no direct solution exists and we must use iterative methods to obtain the solution. It is possible to use bound constrained version of optimization algorithms such as conjugate gradients or limited memory variable metric methods (Schwartz and Polak, 1997 Thiebaut, 2002) but multiplicative methods have also been derived to enforce non-negativity and deserve particular mention because they are widely used RLA (Richardson, 1972 Lucy, 1974) for Poissonian noise and ISRA (Daube-Witherspoon and Muehllehner, 1986) for Gaussian noise. 

Goldfard, D., "Factorized Variable Metric Methods for Unconstrained Optimization", Mathematics of Computation, 30 (136) 796-811 (1976). 

Dixon, L. C. W. and L. James. On Stochastic Variable Metric Methods. In Analysis and Optimization of Stochastic Systems. Q. L. R. Jacobs et al. eds. Academic Press, London (1980). 

Goldfarb, D. A Family of Variable Metric Methods Derived by Variational Means. Math Comput 24 23-26(1970). 

Powell, M. J. D., The convergence of variable metric methods for nonlinearly constrained optimization calculations, in Nonlinear Programming 3 (Mangasarian, O. L., Meyer, R., Robinson, S.. eds.). Academic Press, New York, 1978. 

The method (also called variable metric method, ref.21) is based on the correction formula... 

Goldfarb, D. (1970). A family of variable-metric methods derived by variational means, Math. Comput. 24, 23-26. 

The quasi-Newton or variable-metric methods introduced by Davidon 1 have now become the standard methods for finding an unconstrained minimum of a differentiable function f(x), and an excellent review of the basic theory has been given by Dennis and Mord f2. ... 

Davidon, W.C., "Variable Metric Method for Minimization", A.E.C. Research and Development Report 1959, ANL-5990. 

Goldfarb,D., "Extension of Davidon s Variable Metric Method... 

There are many variants of this kind of algorithm and examples of some of them may be found in chapter 4 of ref. 8. It should also be pointed out that such methods may be combined with those variable metric methods which estimate A-1, so that instead of calculating A 1 at every stage, an estimate of it may be obtained merely by updating the previously calculated matrix. Some examples of studies undertaken by such a combined method may be found in the review by Yde.27... 

However, in a quantum chemical context there is often one overwhelming difficulty that is common to both Newton-like and variable-metric methods, and that is the difficulty of storing the hessian or an approximation to its inverse. This problem is not so acute if one is using such a method in optimizing orbital exponents or internuclear distances, but in optimizing linear coefficients in LCAO type calculations it can soon become impossible. In modern calculations a basis of say fifty AOs to construct ten occupied molecular spin-orbitals would be considered a modest size, and that would, even in a closed-shell case, give one a hessian of side 500. In a Newton-like method the problem of inverting a matrix of such a size is a considerable... 

In summary, therefore, there is too little work with Newton-like methods to make any assertion about their utility in quantum chemistry, but there is enough work with variable-metric methods to make it possible to assert with some confidence that they are worth very serious consideration by any worker wishing to optimize orbital exponents or nuclear positions in a wavefunction. 

Non-linear programming is a fast growing subject and much research is being done and many new algorithms appear every year. It seems to the Reporters that the current area of major interest in the field is the area of variable-metric methods, particularly those not needing accurate linear searches. Unfortunately, from a quantum chemical point of view, such methods are liable to be of use only in exponent and nuclear position optimization and in this context, as we have seen, Newton-like methods are also worth serious consideration. 

W. C. Davidon, Variable Metric Method for Minimization, Report ANL-5990 (rev.), Argonne National Laboratory, Argonne, 111., 1959. 

The Quasi-Newton, or variable metric, methods of optimization... 

Hessian or " Eigenvector Following (EF), depending on how A is chosen. We wili p pseudo-Newton-Raphson or variable metric methods. It is clear that they do not... 

To obtain equilibrium geometries for small molecules and clusters we have implemented a variable metric method which is based on a quasi-Newton scheme and is widely used in optimization theory (Lipkowitz and Boyd 1993 Schlegel 1987). In this... 

The first generally successful method was presented by Davidon (1959), who called it a variable metric method. It was subsequently developed by Fletcher and Powell (1963). They show, and it is straightforward to verify directly, that the following rank two correction satisfies the basic recursion requirement. 

---