Gradient-type methods

LI Method of steepest deseent Consi(k r again the invt. ise ])robl( m 

Wc start our discussion with the nrost important and clc arly understandable method of. stee[)( st dc seent. The idea of this method can be cx[)lained using the csxarnplc of misfit functional minimization  

In the last formula, pp is some given metric in a Hilbert space D  

Note that, in general cases, the misfit functional (5.2) may have several minima. We will distinguish three types of minimums strong local minimum, weak local minima, and global minimum. 

Definition 23 The vector m , is a strong local minimum of the functional 0(m), if there exists 6 0 such that 

---

To improve the convergence of the gradient-type method, Tannor et al. [81, 93] suggested employing the Krotov iteration method [102]. In formulating their method, they utilize a penalty function of the form /[e(f)] = pe (f). In Tannor s Krotov method, the fcth iteration step of the solution process is given by... 

It was reported that the convergence of the Krotov iteration method [81, 93] was four or five times faster than that of the gradient-type methods. The formulation of Rabitz and others, [44, 45, 92], designed to improve the convergence of the above algorithm, introduces a further nonlinear propagation step into the adjoint equation (i.e., the equation for the undetermined Lagrange multiplier % t)) and is expressed as... 

Conjugate gradient-type methods form a class of minimization procedures that accomplish two objectives ... 

R. W. Freund, SIAM J. Sci. Stat. Comput., 13,425 (1992). Conjugate Gradient-Type Methods... 

Regularized gradient-type methods in the solution of nonlinear inverse problems... 

Therefore, the problem of minimizing the parametric functional, given by equation (5.117), can be treated in a similar way to the minimization of the conventional Tikhonov functional. The only difference is that now we introduce some variable weighting matrix Wg for the model parameters. The minimization problem for the parametric functional introduced by equation (5.117) can be solved using the ideas of traditional gradient-type methods. 

Kawata et al [178] proposed to update the DIIS basis by the the modified Broyden (MB) multidimensional secant method [179, 180]. However, it does not seem that in general the MB update could offer an essential advantage over the MDIIS procedme. It was pointed out by Hamilton and Pulay [175] that the DIIS technique is very close to conjugate gradient type methods. Therefore, the successive DIIS minimization by itself improves the update (4.A.32), as indicated above. 

Freund, R. W. [1992] Conjugate gradient-type methods for linear systems with complex symmetrical coefficient matrices, SIAM J. Set Stat. Comp., 13, 425-448. 

---