Conjugate direction methods

Hestenes, M. R. Conjugate-Direction Methods in Optimization. Springer-Verlag, New York (1980). 

The inversion operator i acts on the electronic coordinates (fr = —r). It is employed to generate gerade and ungerade states. The pre-exponential factor, y is the Cartesian component of the i-th electron position vector (mf. — 1 or 2). Its presence enables obtaining U symmetry of the wave function. The nonlinear parameters, collected in positive definite symmetric 2X2 matrices and 2-element vectors s, were determined variationally. The unperturbed wave function was optimized with respect to the second eigenvalue of the Hamiltonian using Powell s conjugate directions method [26]. The parameters of were... 

There has been little recent work on methods for differentiable functions which avoid explicit evaluation of derivatives. Powell s conjugate direction method 36 is still used, but the generally accepted approach is now to use standard quasi-Newton methods with finite-difference approximations to the derivatives. On the other hand there has been considerable interest in methods for nondifferentiable functions, as shown by the collection of papers edited by Balinski and Wolfe 37, in which the technique described by Lemarechal is of particular interest. Other contributions in this difficult field are due to Shor 38, ... 

There are also a large number of conjugate direction methods which, although using only the function and gradient values, make estimates of A x at each stage and choose the directions of descent according to... 

The nonzero vectors dj,. . . , dj are said to be conjugate with respect to the positive definite matrix H if they are linearly independent and d Hdy = 0 for i A j. A method that generates such directions when applied to a quadratic function with Hessian matrix H is called a conjugate direction method. These methods will locate the minimum of a quadratic function in a finite number of iterations, and they can also be applied iteratively to optimize nonquadratic functions. 

Conjugate direction method, 2552, 2553 Conjugate gradient methods, 2552-2553 Connectionist processing model, see Neural networks... 

An alternative formula for P, is the Polak-Ribiere formula see Leach, p. 225.) The idea of the conjugate-gradient method (which really should be called the conjugate-direction method) is to choose each new step in a direction that is conjugate to the directions used in the previous steps (where the word conjugate has a certain technical... 

Conjugate direction methods are suitable when the problem is large and the Hessian is unavailable because of computational time or memory allocation requirements. Certain hybrid algorithms are useful with very large problems they can exploit the features of conjugate direction methods, also partially using the Hessian (Chapter 4). 

To And the ny conjugate directions, both the Powell and Brent methods require an amount of one-dimensional searches in the order of and thus more expensive than conjugate direction methods that estimate the function gradient. 

Inexact Newton s methods update only a portion of the Hessian and solve the linear system using an iterative method. This family of methods is a hybrid (classical Newton method and conjugate direction method). These methods are useful in solving very large problems (see Chapter 4). 

This chapter deals with the problem of finding the unconstrained minimum of a function P x) that involves the variables x e R with Wv 1. Section 3.4 showed that conjugate direction methods are useful in solving large-scale unconstrained minimization problems. The version that uses the Pollack-Ribiere and Fletcher-Reeves methods sequentially is often particularly effective. 

This chapter describes various versions of Newton s method and hybrid (Newton-conjugate directions) methods that often deliver superior results as thqr make use of certain features of the specific problem. 

The method proposed by Fletcher and Reeves (see Chapter 3) is particularly useful the conjugate directions method was first applied to the solution of linear systems by Hestenes and Stiefel (1952). 

Unconstrained optimization methods are discussed in Chapter 3. Heuristic methods, gradient methods and the conjugate direction methods are introduced together with Newton s method and modified Newton and quasi-Newton methods. Convergence and stop criteria are discussed, implemented in generalized classes, and used to optimize the design and operation of batch and fixed-bed reactors. 

---