Newton-Gauss algorithm Marquardt parameter

The best-fitting set of parameters can be found by minimization of the objective function (Section 13.2.8.2). This can be performed only by iterative procedures. For this purpose several minimization algorithms can be applied, for example, Simplex, Gauss-Newton, and the Marquardt methods. It is not the aim of this chapter to deal with non-linear curve-fitting extensively. For further reference, excellent papers and books are available [18]. 

In general, the error e tic-q-i+j, 0) is a non-linear function of the parameter vector 0. Therefore, the above problem is a well-known nonlinear least squares problem (NLSP) that may be solved by various optimisation algorithms such as the Levenberg-Marquardt algorithm [2], the quasi-Newton method or the Gauss-Newton (GN) algorithm [3]. 

When the equations are nonlinear in the parameters, the parameter estimates are obtained by minimizing the objective function by methods like that of Newton-Raphson or that of Newton-Gauss or an adaptation of the latter such as the Marquardt algorithm [1963], In the latter case parameters are iteratively improved by the following formula ... 

If ARRs can be established in symbolic form, they may be differentiated analytically with respect to the parameters to be estimated. That is, the Jacobian of the cost function to be minimised can be provided and a gradient based method such as the Gauss-Newton, or the Levenberg-Marquardt algorithm can be used. As a result, the number of iteration is much lower in comparison to a gradient free algorithm such as the one of Nelder-Mead which is important in real-time FDI. 

---