Quadratic models

Most gradient optimization methods rely on a quadratic model of the potential surface. The minimum condition for the... 

In simple relaxation (the fixed approximate Hessian method), the step does not depend on the iteration history. More sophisticated optimization teclmiques use infonnation gathered during previous steps to improve the estimate of the minunizer, usually by invoking a quadratic model of the energy surface. These methods can be divided into two classes variable metric methods and interpolation methods. 

Model. Owing to the curvature of velocity with depth, a quadratic model was specified ... 

Fig. 3.12. Quadratic model based on (3.14.5.2) with substrate inhibition at an agitation speed of 200 lpm and...

X-axis. It presents the coefficients of the linear models (straight lines) fitted to the several curves of Figure 67-1, the coefficients of the quadratic model, the sum-of-squares of the differences between the fitted points from the two models, and the ratio of the sum-of-squares of the differences to the sum-of-squares of the X-data itself, which, as we said above, is the measure of nonlinearity. Table 67-1 also shows the value of the correlation coefficient between the linear fit and the quadratic fit to the data, and the square of the correlation coefficient. 

This section describes the basic idea of least squares estimation, which is used to calculate the values of the coefficients in a model from experimental data. In estimating the values of coefficients for either an empirical or theoretically based model, keep in mind that the number of data sets must be equal to or greater than the number of coefficients in the model. For example, with three data points of y versus x, you can estimate at most the values of three coefficients. Examine Figure 2.7. A straight line might represent the three points adequately, but the data can be fitted exactly using a quadratic model... 

Fit a quadratic model with these data and determine the value of x that maximizes the yield. 

Solution. The quadratic model is y = + 0 + 03x2. The estimated coefficients... 

Fit the full second-order (quadratic) model to the data. 

Find df(x)/dx = 0, a stationary point of the quadratic model of the function. The result obtained by differentiating Equation (5.6) with respect to x is... 

Trust regions. The name trust region refers to the region in which the quadratic model can be trusted to represent /(x) reasonably well. In the unidimensional line search, the search direction is retained but the step length is reduced if the Newton step proves to be unsatisfactory. In the trust region approach, a shorter step length is selected and then the search direction determined. Refer to Dennis and Schnabel (1996) and Section 8.5.1 for details. 

The trust region approach estimates the length of a maximal successful step from xk. In other words, x < p, the bound on the step. Figure 6.11 shows /(x), the quadratic model of /(x), and the desired trust region. First, an initial estimate of p or the step bound has to be determined. If knowledge about the problem does... 

Representation of the trust region to select the step length. Solid lines are contours of fix). Dashed lines are contours of the convex quadratic approximation of fix) at x. The dotted circle is the trust region boundary in which 8 is the step length. x0 is the minimum of the quadratic model for which H(x) is positive-definite. 

In some cases when estimates of the pure-error mean square are unavailable owing to lack of replicated data, more approximate methods of testing lack of fit may be used. Here, quadratic terms would be added to the models of Eqs. (32) and (33), the complete model would be fitted to the data, and a residual mean square calculated. Assuming this quadratic model will adequately fit the data (lack of fit unimportant), this quadratic residual mean square may be used in Eq. (68) in place of the pure-error mean square. The lack-of-fit mean square in this equation would be the difference between the linear residual mean square [i.e., using Eqs. (32) and (33)] and the quadratic residual mean square. A model should be rejected only if the ratio is very much greater than the F statistic, however, since these two mean squares are no longer independent. 

The methods used were those of Mitchell ( 1 ), Kurtz, Rosenberger, and Tamayo ( 2 ), and Wegscheider T ) Mitchell accounted for heteroscedastic error variance by using weighted least squares regression. Mitchell fitted a curve either to all or part of the calibration range, using either a linear or a quadratic model. Kurtz, et al., achieved constant variance by a... 

In reference 88, response surfaces from optimization were used to obtain an initial idea about the method robustness and about the interval of the factors to be examined in a later robustness test. In the latter, regression analysis was applied and a full quadratic model was fitted to the data for each response. The method was considered robust concerning its quantitative aspect, since no statistically significant coefficients occurred. However, for qualitative responses, e.g., resolution, significant factors were found and the results were further used to calculate system suitability values. In reference 89, first a second-order polynomial model was fitted to the data and validated. Then response surfaces were drawn for... 

Based on the obtained response surface, a second roimd of optimization follows, using the steepest ascent method where the direction of the steepest slope indicates the position of the optimum. Alternatively, a quadratic model can be fitted around a region known to contain the optimum somewhere in the middle. This so-called central composite design contains an imbedded factorial design with centre... 

A quadratic equation was found to fit the crushing strength values the best compared to other Scheffe models [31]. The calculated coefficients of the quadratic model equation (21) are presented in Table 4.4. 

The complete linear models were determined by regression (Table 9). The R2 values for In Y2, T4, In Y5 In Y6 and mainly for Y3 are quite low, indicating that the linear model is inadequate for describing the situation for these variables and that a quadratic model could better fit the data. Nevertheless, Y6 from experiment 12 is a good value, as well as the other responses in this trial and may produce the optimum. 

When the standard curve has been established and the LLOQ and ULOQ validated, the assessment of unknown concentrations by extrapolation is not allowed beyond the validated range. The most accurate and precise estimates of concentration is in the linear portion of the curve even if acceptable quantitative results can be obtained up to the boundary of the curve using a quadratic model. For a linear model, statistic calculations suggest a minimum of six concentrations evenly placed along the entire range assayed in duplicate [5,7,8]. 

We discuss in this section several models used in optimizations. Of these, the most successful are the quadratic model and its modifications, the restricted second-order model and the rational function model. 

The problem with the linear model is that it gives no information about the curvature of fix). This is provided by the more useful local quadratic model... 

To determine the stationary points of the quadratic model we differentiate the model and set the result equal to zero. We obtain a linear set of equations... 

The quadratic model is an improvement on the linear model since it gives information about the curvature of the function and contains a stationary point. However, the model is still unbounded and it is a good approximation to fix) only in some region around xc. The region where we can trust the model to represent fix) adequately is called the trust region. Usually it is impossible to specify this region in detail and for convenience we assume that it has the shape of a hypersphere s <, h where h is the trust... 

To summarize, in the RF approach we make the quadratic model bounded by adding higher-order terms. This introduces n+1 stationary points, which are obtained by diagonalizing the augmented Hessian Eq. (3.22). The figure below shows three RF models with S equal to unity, using the same function and expansion points as for the linear and quadratic models above. Each RF model has one maximum and one minimum in contrast to the SO models that have one stationary point only. The minima lie in the direction of the true minimizer. 

It is often desirable that the approximate Hessian is positive definite so that the quadratic model has a minimum. To ensure this we may use the Broyden-Fletcher-Goldfarb-Shanno (BFGS) update given by... 

If the quadratic model or one of its modifications are used, the local region presents no difficulties. All methods converge rapidly since they effectively reduce to Newton s method in the local region. In this section we briefly discuss local convergence rates and stopping criteria. 

Global strategies for minimization are needed whenever the current estimate of the minimizer is so far from x that the local model is not a good approximation to fix) in the neighborhood of x. Three methods are considered in this section the quadratic model with line search, trust region (restricted second-order) minimization and rational function (augmented Hessian) minimization. 

---