The Marquardt Method

Marquardt [9] has developed an interpolation technique between the Gauss-Newton and the steepest descent methods. This interpolation is achieved by adding the diagonal matrix (XI) to the matrix (J J) in Eq. (7.168)  

The value of X is chosen, at each iteration, so that the corrected parameter vector will result in a lower sum of squares in the following iteration. It can be easily seen that when the value of X is small in comparison with the elements of matrix (J J), the Marquardt method approaches the Gauss-Newton method when X is very large, this method is identical to steepest descent, with the exception of a scale factor that does not affect the direction of the parameter correction vector but that gives a small step size. 

According to Marquardt, it is desired to minimize in the maximum neighborhood over which the linearized function will give adequate representation of the nonlinear function. 

Therefore, the method for choosing X must give small values of A, when the Gauss-Newton method would converge efficiently and large values of X when the steepest descent method is necessary. 

The Marquardt method may likewise be applied to Newton s method. In this case, the diagonal matrix XI is added to the Hessian matrix in Eq. (7.173)  

---

The best-fitting set of parameters can be found by minimization of the objective function (Section 13.2.8.2). This can be performed only by iterative procedures. For this purpose several minimization algorithms can be applied, for example, Simplex, Gauss-Newton, and the Marquardt methods. It is not the aim of this chapter to deal with non-linear curve-fitting extensively. For further reference, excellent papers and books are available [18]. 

The optimization can be carried out by several methods of linear and nonlinear regression. The mathematical methods must be chosen with criteria to fit the calculation of the applied objective functions. The most widely applied methods of nonlinear regression can be separated into two categories methods with or without using partial derivatives of the objective function to the model parameters. The most widely employed nonderivative methods are zero order, such as the methods of direct search and the Simplex (Himmelblau, 1972). The most widely used derivative methods are first order, such as the method of indirect search, Gauss-Seidel or Newton, gradient method, and the Marquardt method. 

This particular NLLSQ program used the Marquardt method of expanding a model in a truncated Taylor s type series and solves for improved estimates of parameters in an iterative manner. The very nature of this strategy is that it finds a particular direction to move in search of better parameter estimates which is not exactly like the normal truncated Taylor s series nor that of the direction of steepest descents". 

For normally distributed errors the parameter region in which = constant can give boundaries of the confidence limits. The value of a obtained in the Marquardt method gives the minimum x in- we set yj" = Xmin + some A% and then look at contours in parameter... 

Non-linear least-squares fitting by the Marquardt method [19,20] appears to be the most commonly used technique for hiexponential fluorescence decay analysis, at least for a time-domain measurement such as used here [21,22]. Fitting by this method requires evaluation of the derivatives of the model equation (Equation... 

The model parameters were adjusted through the Marquardt method (9), for each temperature, and finm the values of ki and k2 the Arrhenius equation constants were determined. Their values are... 

Next, the average value of the frequency factor ko was established, as well as the activation energy E, from the Arrhenius equation - based on the calculations of the average constant rate for experiments conducted in various temperatures, followed by calculations for all sets of data, while values for ko and E estimated in the previous step were used as a staring point. The Marquardt method was used to minimise the function (4). At this stage the statistical analysis of the results was also conducted. The algorithm presented in [1] was used. 

The Marquardt method was employed to minimise the value of the function (2) At this stage, a statistical analysis of the results was also performed using the algorithm presented in reference [2],... 

Eq. (7.178) gives the correction of the parameter vector when fitting multiple dependent variables simultaneously. Eq. (7.178) becomes identical to Eq. (7.168) when v = I, that is, when only one dependent variable is fitted. When using the Marquardt method, the correction of the parameter vector is calculated from... 

Example 7.1 Nonlinear Regression Using the Marquardt Method. In Prob. 5.5, we described the kinetics of a fermentation process that manufactures penicillin antibiotics. When the microorganism Penicillium chrysogenum is grown in a batch fermentor under carefully controlled conditions, the cells grow in a rate that can be modeled by the logistic law... 

The experimental data in Table E7.1 were obtained from two penicillin fermentation runs conducted at essentially identical operating conditions. Using the Marquardt method, fit the above two equations to the experimental data and determine the values of the parameters I , b2, 3, and K which minimize the weighted sum of squared residuals. 

Method of Solution The Marquardt method using the Gauss-Newton technique, described in Sec. 7.4.4, and the concept of multiple nonlinear regression, covered in Sec. 7.4.5, have been combined together to solve this example. Numerical differentiation by forward finite differences is used to evaluate the Jacobian matrix defined by Eq. (7.164). 

NLR.m This function evaluates the fitting parameters by the Marquardt method. At the beginning, the function examines the length of the input arguments and sets the default value, if necessary. The experimental independent and dependent variables should be introduced to the function by matrices of the same size (column vectors in the case of single independent... 

Example 7.1 Nonlinear Regression Using the Marquardt Method Table E7.1 Experimental data for penicillin fermentation 503... 

Example 7.1 Nonlinear Regression Using the Marquardt Method... 

---