Marquardt’s method

Show how to make the Hessian matrix of the following objective function positive-definite at x = [1 l]r by using Marquardt s method ... 

The equations are solved for an assumed set of parameters, P = [e, De, pj, ka, ks, 0jj], using finite difference equations, which are described in standard texts on the subject (32). The vector of unknown parameters P is determined by minimizing the mean square relative error between the model-predicted and experimental breakthrough curves. Minimization of the mean-square relative errors was obtained using Marquardt s method (33). 

Sanderson and Chien (18) solve Equations (7), (8), and (13) to determine compositions of vapor and liquid phases in chemical and phase equilibrium given temperature and pressure. A set of independent chemical reactions is selected with guesses for extent of reaction. Solution of Equation (13) leads to compositions in phase equilibrium, but applies only for a vapor and liquid in equilibrium. Residuals of Equations (7) and (8) are computed and extents of reaction, g, and moles of species j, n, are adjusted using Marquardt s method (15). 

Many variations of the correction method we have proposed can be used. Among these are the use of the same Jacobian for several iterations and the use of modified Newton methods such as Marquardt s method (8). We have tried Marquardt s method on some of these problems without observing any significant improvement, but this is only a tentative evaluation. Improved methods for generating starting conditions would be helpful. 

Another important factor to consider is the objective function used in the fit. The best results seem to be obtained by minimizing the following objective function, using Marquardt s method [162] ... 

We have found on using Marquardt s method for solving a complicated chemical equilibrium, however, that it offers no substantial improvement. Therefore we refrain from a detailed description of the method. 

Derivative methods also exist, in which the set (A2.1) is solved by searching for the minimum of the function CD. These include the gradient method and Marquardt s method, which have been discussed in section 5.5.2 from the point of view of chemical equilibrium calculations. 

The minimization of SLG(k,co) can now be accomplished by applying the Gauss-Newton method with Marquardt s modification and a step-size policy as described in earlier chapters. 

As mentioned in Chapter 4, although this is a dynamic experiment where data are collected over time, we consider it as a simple algebraic equation model with two unknown parameters. The data were given for two different conditions (i) with 0.75 g and (ii) with 1.30 g of methanol as solvent. An initial guess of k =1.0 and k2=0.01 was used. The method converged in six and seven iterations respectively without the need for Marquardt s modification. Actually, if Mar-quardt s modification is used, the algorithm slows down somewhat. The estimated parameters are given in Table 16.1 In addition, the model-calculated values are... 

In this work, we first regressed the isothermal data. The estimated parameters from the treatment of the isothermal data are given in Table 16.6. An initial guess of (ki=l.O, k2=1.0, k3=1.0) was used for all isotherms and convergence of the Gauss-Newton method without the need for Marquardt s modification was achieved in 13, 16 and 15 iterations for the data at 375, 400, and 425°C respectively. 

In this example the number of measured variables is less than the number of state variables. Zhu et al. (1997) minimized an unweighted sum of squares of deviations of calculated and experimental concentrations of HPA and PD. They used Marquardt s modification of the Gauss-Newton method and reported the parameter estimates shown in Table 16.24. 

Using an initial guess of kj=350 and k2=l the Gauss-Newton method converged in five iterations without the need for Marquardt s modification. The estimated parameters are k,= 334.27 2.10% and k2=0.38075 5.78%. The model-calculated values are compared with the experimental data in Table 17.1. As seen the agreement is very good in this case. 

Indeed, using the Gauss-Newton method with an initial estimate of k(0)=(450, 7) convergence to the optimum was achieved in three iterations with no need to employ Marquardt s modification. The optimal parameter estimates are k = 420.2 8.68% and k2= 5.705 24.58%. It should be noted however that this type of a model can often lead to ill-conditioned estimation problems if the data have not been collected both at low and high values of the independent variable. The convergence to the optimum is shown in Table 17.5 starting with the initial guess k(0)=(l, 1). 

Equation 17.10 can now be used to obtain the two unknown parameters (kLa and Cq2 ) by fitting the data from the gassing-in period of the experiment. Indeed, using the Gauss-Newton method with an initial guess of (10, 10) convergence is achieved in 7 iterations as shown in Table 17.6. There was no need to employ Marquardt s modification. The FORTRAN program used for the above calculations is also provided in Appendix 2. 

If fix) is convex, H(x) is positive-semidefinite at all points x and is usually positive-definite. Hence Newton s method, using a line search, converges. If fix) is not strictly convex (as is often the case in regions far from the optimum), H(x) may not be positive-definite everywhere, so one approach to forcing convergence is to replace H(x) by another positive-definite matrix. The Marquardt-Levenberg method is one way of doing this, as discussed in the next section. 

Two adjustable parameters of fhe equafions can be found by an optimization technique using Marquardt s or Rosenbrock s maximum likelihood method of minimizafion... 

The steepest descent method is very effective far from the minimum of , but is always much less efficient than the Gauss-Newton method near the minimum of . Marquardt (1963) has proposed a hybrid method that combines the advantages of both Gauss-Newton and steepest descent methods. Mar-quardt s method, combined with the Hellmann-Feynman pseudolinearization of the Hamiltonian energy level model, is the method of choice for most nonlinear molecular spectroscopic problems. 

---