Variable metric

In simple relaxation (the fixed approximate Hessian method), the step does not depend on the iteration history. More sophisticated optimization teclmiques use infonnation gathered during previous steps to improve the estimate of the minunizer, usually by invoking a quadratic model of the energy surface. These methods can be divided into two classes variable metric methods and interpolation methods. 

Owing to the constraints, no direct solution exists and we must use iterative methods to obtain the solution. It is possible to use bound constrained version of optimization algorithms such as conjugate gradients or limited memory variable metric methods (Schwartz and Polak, 1997 Thiebaut, 2002) but multiplicative methods have also been derived to enforce non-negativity and deserve particular mention because they are widely used RLA (Richardson, 1972 Lucy, 1974) for Poissonian noise and ISRA (Daube-Witherspoon and Muehllehner, 1986) for Gaussian noise. 

Quasi-Newton or Variable Metric or Secant Methods... 

Goldfard, D., "Factorized Variable Metric Methods for Unconstrained Optimization", Mathematics of Computation, 30 (136) 796-811 (1976). 

Davidon, C.W. Variable Metric Methodfor Minimization, Argonne National Laboratory, Argonne... 

Dixon, L. C. W. and L. James. On Stochastic Variable Metric Methods. In Analysis and Optimization of Stochastic Systems. Q. L. R. Jacobs et al. eds. Academic Press, London (1980). 

Fletcher, R. A New Approach to Variable Metric Algorithms. Comput J13 317 (1970). 

Goldfarb, D. A Family of Variable Metric Methods Derived by Variational Means. Math Comput 24 23-26(1970). 

Powell, M. J. D., The convergence of variable metric methods for nonlinearly constrained optimization calculations, in Nonlinear Programming 3 (Mangasarian, O. L., Meyer, R., Robinson, S.. eds.). Academic Press, New York, 1978. 

The method (also called variable metric method, ref.21) is based on the correction formula... 

The general line-element expression (9.28) allows one to envision possible geometries with fto/i-Euclidean metric [i.e., failing to satisfy one or more of the conditions (9.27a-c)] or with variable metric [i.e., with a matrix M that varies with position in the space, M = M( i )> a Riemannian geometry that is only locally Euclidean cf. Section 13.1]. However, for the present equilibrium thermodynamic purposes (Chapters 9-12) we may consider only the simplest version of (9.28), with metric elements (R R,-) satisfying the Euclidean requirements (9.27a-c). 

Goldfarb, D. (1970). A family of variable-metric methods derived by variational means, Math. Comput. 24, 23-26. 

Equation (2.11) with variable metric tensor describes the invariance in the gravitational case which is characterized by curved space-time. The summation extends over all values of y, and u, so that the sum consists of 4 x 4 terms, of which 12 are equal in pairs, hence 10 independent functions. The motion of a free material point in this field will take the form of curvilinear non-uniform motion. If the matrix of the metric tensor can be diagonalized it is independent of position and the corresponding geometry is said to be flat, which is the special case of SR. 

The quasi-Newton or variable-metric methods introduced by Davidon 1 have now become the standard methods for finding an unconstrained minimum of a differentiable function f(x), and an excellent review of the basic theory has been given by Dennis and Mord f2. ... 

In 1967, Goldfarb 72 improved the method by using the projected gradients in Davidon s variable-metric formula to generate the steps. In 1968, Murtagh and Sargent 73 proposed a "variable-metric projection" version of Rosen s method, using the... 

Davidon, W.C., "Variable Metric Method for Minimization", A.E.C. Research and Development Report 1959, ANL-5990. 

Self-scaling Variable Metric Algorithms", Math. Programming, 1976, 10, (1), 70-90. 

Goldfarb,D., "Extension of Davidon s Variable Metric Method... 

Another technique for handling the mass balances was introduced by Castillo and Grossmann (1). Rather than convert the objective function into an unconstrained form, they implemented the Variable Metric Projection method of Sargent and Murtagh (32) to minimize Gibbs s free energy. This is a quasi-Newton method which uses a rank-one update to the approximation of H l, with the search direction "projected" onto the intersection of hyperplanes defined by linear mass balances. 

There are many variants of this kind of algorithm and examples of some of them may be found in chapter 4 of ref. 8. It should also be pointed out that such methods may be combined with those variable metric methods which estimate A-1, so that instead of calculating A 1 at every stage, an estimate of it may be obtained merely by updating the previously calculated matrix. Some examples of studies undertaken by such a combined method may be found in the review by Yde.27... 

---