Steepest descents algorithm

The steepest descent method, proposed by Cauchy in 1847 [8], is also known as gradient method. It is one of the oldest and simplest first-order algorithms for mini- 

The basic idea is that the gradient vector of the objective function, VI/(0) = [91//36 i dU/d0Nf], represents the direction of faster increase of the function. Hence, the estimate at step s + 1 can be computed via the recursive law 

---

The gradient at the minimum point obtained from the line search will be perpendicular to the previous direction. Thus, when the line search method is used to locate the minimum along the gradient then the next direction in the steepest descents algorithm will be orthogonal to the previous direction (i.e. gk Sk-i = 0)-... 

Fig. 5.29 Method for correcting the path followed by a steepest descents algorithm to generate the intrinsic reaction coordinate. The solid line shows the real path and the dotted line shows the algorithmic approximation to it. (Figure redrawn from Gonzalez C and H B Schlegel 1988. An Improved Algorithm for Reaction Path Following. Journal of Chemical Physics 90 2154-2161.)...

FIGURE 18.1 Illustration of how the steepest descent algorithm follows a path that oscillates around the minimum energy path. 

This approach is useful when dealing with relatively simple partial differential equation models. Seinfeld and Lapidus (1974) have provided a couple of numerical examples for the estimation of a single parameter by the steepest descent algorithm for systems described by one or two simultaneous PDEs with simple boundary conditions. 

The steepest descent algorithm can be summarized in the following steps ... 

Equipped with such a method to estimate the gradient of the energy with respect to the parameters, we can describe a simple steepest descent algorithm that tries to optimize the parameters in a single VMC run. 

Fig. 3.6. Steepest descent algorithm (thin line) The derivative vector from the initial point Pq (Xq./q) defines the line search direction. The derivative vector does not point directly toward the minimum (O). The negative gradient of the...

The steepest descent algorithm is sure-fire. If the line minimization is carried out sufficiently accurately, it will always lower the function value, and is therefore guaranteed to approach a minimum. It has, however, two main problems. Two subsequent line searches are necessarily perpendicular to each other if there was a gradient component along the previous search direction, the energy could be further lowered in this direction. The steepest descent algorithm therefore has a tendency for each line search to partly spoil the function lowering obtained by the previous search. The steepest... 

---