Levenberg—Marquardt methods

The Levenberg-Marquardt Method described in this section represents a compromise between the Gauss-Newton Method described in Section 19.4.1 and the Method of Steepest Descent described in Section 19.4.2. The Method of Steepest Descent is used far from the converged value, moving smoothly to the Gauss-Newton Method as the solution is approached. 

The critical concepts encompassed by the Levenberg-Marquardt Method are the selection of the scaling factor for the Method of Steepest Descent and an approach for making a smooth transition from one method to the other. The curvature matrix a is replaced by cc such that 

When A is large, a is diagonally dominant, and the method approaches that described in equation (19.31). When A is small, the method approaches that described in equation (19.31). 

---

The Levenberg-Marquardt method is used when the parameters of the model appear nonlinearly (Ref. 231). We stiU define... 

The second term in the second derivative is dropped because it is usually small [remember that will be close to y xi, a)]. The Levenberg-Marquardt method then iterates as follows... 

Modifications of Levenberg-Marquardt Method Fletcher s Modification... 

A number of modifications to eliminate some less favorable aspects of the Levenberg-Marquardt method were considered by Fletcher. For instance, the arbitrary initial choice of the adjustable parameter A, if poor, can cause an excessive number of evaluations of squared error, before a realistic value is obtained. This is especially noticeable if v, i.e., J R x), is chosen to be small, i.e., v = 2. Another disadvantage of the method is that the reduction of A to v at the start of each iteration may also cause excessive evaluations, especially when V is chosen to be large, i.e., = 10. The... 

Solver for non-linear data fitting tasks. Several examples are based on the fitting tasks already solved by the Newton-Gauss-Levenberg/Marquardt method in the earlier parts of this chapter. 

The Levenberg-Marquardt method is used to model data with non-linear dependencies. Here, we may have m functions /i, /2, fz-.-fm that depend on n parameters pi,P2---pn written in vector form as... 

We perform nonlinear fitting using the Levenberg-Marquardt method implemented in the MRQMIN routine [75], From the experimental end, eight families of data are involved, namely, x (T) and x jT) at four frequencies, taken from Ref. 64. From the theory end, we employ formulas (4.121)-(4.124) with the numerical dispersion factors. The results of fitting are presented in Figures 4.6 and 4.7 and Table I. 

Nonlinear least-squares fitting in the time domain (Levenberg-Marquardt method)... 

The procedure was tested on simulated time domain MRS data where the model data consisted of metabolite peaks at 3.2, 3.0 and 2.0 ppm representing choline, creatine and IV-acetylaspartate (NAA) respectively, with corresponding values of Ak of 1.0, 1.0 and 3.0 units.89 White noise of specified standard deviation, crt, was then added. The Levenberg-Marquardt method requires suitable initial values for each of the nine parameters being fitted. The initial values of the three frequencies were taken as their known values. An exponentially decaying curve with a constant offset parameter was fitted, using a nonlinear least-squares fit, to the envelope of the free induction decay, Mv(t), in order to obtain an initial value for T and for the amplitudes, each of which was taken to be one-third of the amplitude of the envelope. The constant offset was added to account for the presence of the noise. 

A nonlinear, multiparameter regression procedure (Levenberg-Marquardt method) was applied to estimate the kinetic parameters involved in Equations (51)-(54). The experimental concentrations of the pollutant (4-CP) and of the main intermediate species (4-CC and HQ) at different reaction times were compared with model predictions. Under the operating conditions of the experimental runs, it was found that the terms a Ci- cp(f)/ aiC4-cc(f)/ and 02CHQ(t) were much lower than 1. As a result, the final expressions employed for the regression of the kinetic parameters are the following ... 

---