Termination Criterion, Numerical Derivatives

And now we introduce numerical derivatives. In the example above, we used explicit formulas for the derivatives of the residuals with respect to the parameters. Often it is not easy, or even impossible, to work out the correct equations. Numerical computation of the derivatives is always possible. Usually it is slower and also numerically less accurate. The general formula is  

This is a rather casual notation and we need to clarify what is meant. p+Ap is a new parameter vector with only the i-th parameter p shifted by the small amount Apu In Main NG2, m, Ap is calculated as lxlO 4 p . The factor lxlCk4 is somewhat arbitrary and experimentation is usually the best way of 

The Matlab program Main NG2. m has implemented the additions for a termination criterion and numerical derivatives. Refer to the Matlab Help Desk for information on the while end loop and also the break command. 

Marquardt, based on ideas by Levenberg, suggested a very elegant and efficient method to manage the problems associated with divergence. 

The pseudo-inverse for the calculation of the shift vector in equation (4.67) has been computed traditionally as J+= (J Jp1 J. Adding a certain number, the Marquardt parameter mp, to the diagonal elements of the square matrix J J prior to its inversion, has two consequences (a) it shortens the shift vector 8p and (b) it turns its direction towards steepest descent. The larger the Marquardt parameter, the larger is the effect. In matrix formulation, we can write  

---