Gradient linear model

McIntosh, A. Fitting Linear Models An Application of Conjugate Gradient Algorithms, Springer-Verlag, New York (1982). 

Fig. 4 Predicted versus observed summer Anoxic Factor (AF) in (a, b) Foix Reservoir (Spain), (c, d) San Reservoir (Spain), (e, f) Brownlee Reservoir (USA), and (g, h) Pueblo Reservoir (USA). The results have been arranged to place the systems along a gradient of relative human impact (Foix Reservoir at the top, Pueblo Reservoir at the bottom). Predictions are based on linear models using different independent variables (in brackets) Inflow = streamflow entering the reservoir during the period DOCjjiflow = mean summer river DOC concentration measured upstream the reservoir CljjjAow = mean summer river CU concentration measured upstream the reservoir and Chlepi = mean summer chlorophyll-a concentration measured in the epilimnion of the reservoir. The symbol after a variable denotes a nonsignificant effect at the 95% level. Solid lines represent the perfect fit, and were added for reference. Modified from Marce et al. [48]...

The well-known Box-Wilson optimization method (Box and Wilson [1951] Box [1954, 1957] Box and Draper [1969]) is based on a linear model (Fig. 5.6). For a selected start hyperplane, in the given case an area A0(xi,x2), described by a polynomial of first order, with the starting point yb, the gradient grad[y0] is estimated. Then one moves to the next area in direction of the steepest ascent (the gradient) by a step width of h, in general... 

The smoothing terms have a thermodynamic basis, because they are related to surface gradients in chemical potential, and they are based on linear rate equations. The magnitude of the smoothing terms vary with different powers of a characteristic length, so that at large scales, the EW term should predominate, while at small scales, diffusion becomes important. The literature also contains non-linear models, with terms that may represent the lattice potential or account for step growth or diffusion bias, for example. 

The first RDA ordination axis scored an eigenvalue of 0.493, while the first CCA axis 0.003. This contrast indicated that the linear model fitted well, but the unimodal model fitted poorly. Thus, only the RDA ordination diagram was shown in Fig. 4. The diagram visualizes the land degradation gradient. Again, the DEF and the GB soils were shown to be the extremes, while the DDF soil the intermediate. Relationships between the changes in... 

The linear model, which may also be constructed from an approximate gradient, is simple but not particularly useful since it is unbounded and has no stationary point. It contains no information about the curvature of the function. It is the basis for the steepest descent method in which a step opposite the gradient is determined by line search vide infra). 

In this way the basic experiment is defined for the linear model, and the gradient that indicates the direction of the fastest response increase or decrease is obtained. When a response maximum or minimum is searched for, the experimental center is moved that way and a new experiment for the linear model performed. The procedure is repeated until moving along the gradient has an effect. When this has no effect, it means we are close to the optimum. Polynomials of higher order, mostly the second, are used in the optimum region. 

It is much more complicated to draw conclusions after an unsuccessful application of the method of movement to optimum along the gradient. Drawing conclusions depends much more on whether the optimum is close by, far away or its position is unknown, and whether the linear model is adequate. Typical cases of such situations are demonstrated in Fig. 2.46. 

If the optimum area is far away and the linear model adequate, there exist good reasons for the method of steepest ascent to be successful. A possible explanation for a failure in applying the gradient method may lie in the form of the response surface with one extreme. The response surface may in reality have the form shown Fig. 2.47. 

In practice, rarely are models of the exact form given by Eq. (7.57) or (7.61) used. The reason is that the scale, or range, and magnitude of the many covariates that may be in a model may be so different that the gradient, which is evaluated during the optimization process, may be unstable. To avoid this instability, cov-ariates are often centered or standardized in the case of linear models or scaled in the case of power or exponential models. For example, if age had a mean of 40 years and a standard deviation of 8 and weight had a mean of 80 kg and standard deviation of 12, then Eq. (7.57) may be standardized using... 

Another model for the electron density gradient was proposed by Blundell and analyzed by Vonk it consists of a linear density change in the interface [21, 28]. In this model, called the geometric linear model, the smoothing function is of rectangular type (Fig. 19.10) and its Fourier transform is given by... 

Figure 19.10 Linear model of the interface gradient. Source Reproduced with permission from Koberstein JT, Morra B, Stein RS. J Appl Crystallogr 1980 13 34 [27], Copyright 1980 lUCr (International Union of Crystallography) (http //dx.doi.org/10.1107/S0021889880011478).

It is noteworthy that the independent variable h is not present at all in this linear model therefore we would obtain the same results by analogous linearization of simple fluid (3.129), cf. end of Sect. 3.6 [9, 23, 24, 45] (but the presence of density gradients will be important in mixtures, see Sect. 4.8). 

GBM, gradient boosting models MLR, multivariate linear regression NN, artificial neural net RF, random forest, n, descriptors, number of descriptors used n, train, size of training set, n, test, size of test set. Fivefold cross-validated results. 

---