Gauss-Newton modification

The above method is the well-known Gauss-Newton method for differential equation systems and it exhibits quadratic convergence to the optimum. Computational modifications to the above algorithm for the incorporation of prior knowledge about the parameters (Bayessian estimation) are discussed in detail in Chapter 8. 

The required modifications to the Gauss-Newton algorithm presented in Chapter 4 are rather minimal. At each iteration, we just need to add the following terms to matrix A and vector b,... 

For models described by a set of ordinary differential equations there are a few modifications we may consider implementing that enhance the performance (robustness) of the Gauss-Newton method. The issues that one needs to address more carefully are (i) numerical instability during the integration of the state and sensitivity equations, (ii) ways to enlarge the region of convergence. 

The above unconstrained estimation problem can be solved by a small modification of the Gauss-Newton method. Let us assume that we have an estimate kw of the parameters at the j iteration. Linearization of the model equation and the constraint around kw yields,... 

With a few minor modifications, the Gauss-Newton method presented in Chapter 4 can be used to obtain the unknown parameters. If we consider Taylor series expansion of the penalty function around the current estimate of the parameter we have,... 

The solution of Equation 10.28 is obtained in one step by performing a simple matrix multiplication since the inverse of the matrix on the left hand side of Equation 10.28 is already available from the integration of the state equations. Equation 10.28 is solved for r=l,...,p and thus the whole sensitivity matrix G(tr,) is obtained as [gi(tHt), g2(t,+1),- - , gP(t,+i)]. The computational savings that are realized by the above procedure are substantial, especially when the number of unknown parameters is large (Tan and Kalogerakis, 1991). With this modification the computational requirements of the Gauss-Newton method for PDE models become reasonable and hence, the estimation method becomes implementable. 

The minimization of SLG(k,co) can now be accomplished by applying the Gauss-Newton method with Marquardt s modification and a step-size policy as described in earlier chapters. 

In this work, we first regressed the isothermal data. The estimated parameters from the treatment of the isothermal data are given in Table 16.6. An initial guess of (ki=l.O, k2=1.0, k3=1.0) was used for all isotherms and convergence of the Gauss-Newton method without the need for Marquardt s modification was achieved in 13, 16 and 15 iterations for the data at 375, 400, and 425°C respectively. 

In this example the number of measured variables is less than the number of state variables. Zhu et al. (1997) minimized an unweighted sum of squares of deviations of calculated and experimental concentrations of HPA and PD. They used Marquardt s modification of the Gauss-Newton method and reported the parameter estimates shown in Table 16.24. 

Using an initial guess of kj=350 and k2=l the Gauss-Newton method converged in five iterations without the need for Marquardt s modification. The estimated parameters are k,= 334.27 2.10% and k2=0.38075 5.78%. The model-calculated values are compared with the experimental data in Table 17.1. As seen the agreement is very good in this case. 

Indeed, using the Gauss-Newton method with an initial estimate of k(0)=(450, 7) convergence to the optimum was achieved in three iterations with no need to employ Marquardt s modification. The optimal parameter estimates are k = 420.2 8.68% and k2= 5.705 24.58%. It should be noted however that this type of a model can often lead to ill-conditioned estimation problems if the data have not been collected both at low and high values of the independent variable. The convergence to the optimum is shown in Table 17.5 starting with the initial guess k(0)=(l, 1). 

Equation 17.10 can now be used to obtain the two unknown parameters (kLa and Cq2 ) by fitting the data from the gassing-in period of the experiment. Indeed, using the Gauss-Newton method with an initial guess of (10, 10) convergence is achieved in 7 iterations as shown in Table 17.6. There was no need to employ Marquardt s modification. The FORTRAN program used for the above calculations is also provided in Appendix 2. 

A second method of improving the Gauss-Newton method is the Marquardt modification. In this case, the equation for P ew is modified by the addition of another term, pi, as in Eq. (21) ... 

---