Gauss-Newton

Kinetic curves were analyzed and the further correlations were determined with a nonlinear least-square-method PC program, working with the Gauss-Newton method. 

As seen in Chapter 2 a suitable measure of the discrepancy between a model and a set of data is the objective function, S(k), and hence, the parameter values are obtained by minimizing this function. Therefore, the estimation of the parameters can be viewed as an optimization problem whereby any of the available general purpose optimization methods can be utilized. In particular, it was found that the Gauss-Newton method is the most efficient method for estimating parameters in nonlinear models (Bard. 1970). As we strongly believe that this is indeed the best method to use for nonlinear regression problems, the Gauss-Newton method is presented in detail in this chapter. It is assumed that the parameters are free to take any values. 

In this chapter we are focusing on a particular technique, the Gauss-Newton method, for the estimation of the unknown parameters that appear in a model described by a set of algebraic equations. Namely, it is assumed that both the structure of the mathematical model and the objective function to be minimized are known. In mathematical terms, we are given the model... 

Minimization of S(k) can be accomplished by using almost any technique available from optimization theory. Next we shall present the Gauss-Newton method as we have found it to be overall the best one (Bard, 1970). 

More elaborate techniques have been published in the literature to obtain optimal or near optimal stepping parameter values. Essentially one performs a univariate search to determine the minimum value of the objective function along the chosen direction (Ak ) by the Gauss-Newton method. 

Formulation of the Solution Steps for the Gauss-Newton Method Two Consecutive Chemical Reactions... 

Equations 4.14 and 4.15 are used to evaluate the model response and the sensitivity coefficients that are required for setting up matrix A and vector b at each iteration of the Gauss-Newton method. 

Finally, an important advantage of the Gauss-Newton method is also that at the end of the estimation besides the best parameter estimates their covariance matrix is also readily available without any additional computations. Details will be given in Chapter 11. 

Starting with the initial guess k(0)=[l, 1, 1]T the Gauss-Newton method easily converged to the parameter estimates within 4 iterations as shown in Table 4.7. In the same table the standard error (%) in the estimation of each parameter is also shown. Bard (1970) also reported the same parameter estimates [0.08241, 1.1330, 2.3437] starting from the same initial guess. 

Table 4.7 Parameter Estimates at Each Iteration of the Gauss-Newton Methodfor Numerical Example-Iwith Initial Guess [1, I, 1]...

The Gauss-Newton method arises when the second order terms on the right hand side of Equation 5.20 are ignored. As seen, the Hessian matrix used in Equation 5.11 contains only first derivatives of the model equations f(x,k). Leaving out the second derivative containing terms may be justified by the fact that these terms contain the residuals e, as factors. These residuals are expected to be small quantities. 

The Gauss-Newton method is directly related to Newton s method. The main difference between the two is that Newton s method requires the computation of second order derivatives as they arise from the direct differentiation of the objective function with respect to k. These second order terms are avoided when the Gauss-Newton method is used since the model equations are first linearized and then substituted into the objective function. The latter constitutes a key advantage of the Gauss-Newton method compared to Newton s method, which also exhibits quadratic convergence. 

Gauss-Newton Method for Ordinary Differential Equation (ODE) Models... 

In this chapter we are concentrating on the Gauss-Newton method for the estimation of unknown parameters in models described by a set of ordinary differential equations (ODEs). 

The above method is the well-known Gauss-Newton method for differential equation systems and it exhibits quadratic convergence to the optimum. Computational modifications to the above algorithm for the incorporation of prior knowledge about the parameters (Bayessian estimation) are discussed in detail in Chapter 8. 

If the dimensionality of the problem is not excessively high, simultaneous integration of the state and sensitivity equations is the easiest approach to implement the Gauss-Newton method without the need to store x(t) as a function of time. The latter is required in the evaluation of the Jacobeans in Equation 6.9 during the solution of this differential equation to obtain G(t). 

THE GAUSS-NEWTON METHOD - NONLINEAR OUTPUT RELATIONSHIP... 

---