Nonlinear steepest descent

The rate of convergence of the Steepest Descent method is first order. The basic difficulty with steepest descent is that the method is too sensitive to the scaling of S(k), so that convergence is very slow and oscillations in the k-space can easily occur. In general a well scaled problem is one in which similar changes in the variables lead to similar changes in the objective function (Kowalik and Osborne, 1968). For these reasons, steepest descent/ascent is not a viable method for the general purpose minimization of nonlinear functions, ft is of interest only for historical and theoretical reasons. 

The methods differ in the determination of the step length factor ak at the Ath iteration, since the direction of the steepest descent is, due to nonlinearities, not necessarily the optimal one, but only for quadratic dependencies. Some methods therefore use the second derivative matrix of the objective function with respect to the parameters, the Hessian matrix, to determine the parameter improvement step-length and its direction ... 

In the case of a nonlinear operator, 4. it is preferable to use an algorithm of the steepest descent method with the quadratic line search. It can bo summarized as follows ... 

The steepest descent method for nonlinear regularized least-squares inversion To solve the problem of minimization of the parametric functional using the steepest descent method, let us calculate the first variation of P (m,d), assuming that the operator A(m) is differentiable, so that... 

In order to complete our description of the LSA method all that remains is to specify Xl and, then, to evaluate the interaction energy. An optimum choice for Xl is determined by the variation condition which yields [10] the local space analogue of a familiar result, namely (RFU + UFR)L = 0. Here F is either the Fock or the Kohn-Sham matrix. Since R, F and U all depend upon Xl, this is a nonlinear relation that must be solved iteratively. The simplest, but least efficient, method of solution is steepest descents which corresponds to the choice... 

The steepest descent method is very effective far from the minimum of , but is always much less efficient than the Gauss-Newton method near the minimum of . Marquardt (1963) has proposed a hybrid method that combines the advantages of both Gauss-Newton and steepest descent methods. Mar-quardt s method, combined with the Hellmann-Feynman pseudolinearization of the Hamiltonian energy level model, is the method of choice for most nonlinear molecular spectroscopic problems. 

The Bird-Carreau model employs the use of four empirical constants (ai, a2, Ai, and A2) and a zero shear limiting viscosity (770) of the solutions. The constants a, az, Ai, and A2, can be obtained by two different methods one method is using a computer program which can combine least square method and the method of steepest descent analysis for determining parameters for the nonlinear mathematical models (Carreau etal, 1968). Another way is to estimate by a graphic method as illustrated in Fig. 20 two constants, Q i and A], are obtained from a logarithmic plot of 77 vs y, and the other two constants, az and A2, are obtained from a logarithmic plot of 77 vs w. 

A simple method, which has been used to arrive at the minimum sum of squares of a nonlinear model, is that of steepest descent. We know that the gradient of a scalar function is a vector that gives the direction of the greatest increase of the function at any point. In the steepest descent method, we take advantage of this property by moving in the opposite direction to reach a lower function value. Therefore, in this method, the initial vector of parameter estimates is corrected in the direction of the negative gradient of O ... 

Linear and nonlinear regression analyses, including least squares, estimated vector of parameters, method of steepest descent, Gauss-Newton method, Marquardt Method, Newton Method,... 

---