Regression non-linear

Non-linear regression calculations are extensively used in most sciences. The goals are very similar to the ones discussed in the previous chapter on Linear Regression. Now, however, the function describing the measured data is non-linear and as a consequence, instead of an explicit equation for the computation of the best parameters, we have to develop iterative procedures. Starting from initial guesses for the parameters, these are iteratively improved or fitted, i.e. those parameters are determined that result in the optimal fit, or, in other words, that result in the minimal sum of squares of the residuals. 

There are a multitude of methods for this task. Those that are conceptually simple usually are computationally intensive and slow, while the fast algorithms have a more complex mathematical background. We start this chapter with the Newton-Gauss-Levenberg/Marquardt algorithm, not because it is the simplest but because it is the most powerful and fastest method. We can t think of many instances where it is advantageous to use an alternative algorithm. 

Because of its relative complexity and tremendous usefulness, we develop the Newton-Gauss-Levenberg/Marquardt algorithm in several small steps and thus examine it in more detail than many of the other algorithms introduced in this book. 

in Chapter 4.4, General Optimisation, we discuss non-linear least-squares methods where the sum of squares is minimised directly. What is meant with that statement is, that ssq is calculated for different sets of parameters p and the changes of ssq as a function of the changes in p are used to direct the parameter vector towards the minimum. 

In this section we demonstrate that it is possible to use the complete vector or matrix of residuals to drive the iterative refinement towards the minimum. 

In chemical engineering, many models are non-linear, which means that they are nonlinear with respect to the parameters. This will be illustrated by a few examples below. The simple straight-line model. 

However, due to the form of the parameters, the following models are non-linear  

In order to separate clearly the hnear and non-linear models, we denote the parameters for the linear models as Pi, and that for the non-linear models as 0,. In this chapter, we discuss the statistical evaluation of non-linear models, which is often very complex therefore, iterative numerical solutions will be used. 

---

A non-linear regression analysis is employed using die Solver in Microsoft Excel spreadsheet to determine die values of and in die following examples. Example 1-5 (Chapter 1) involves the enzymatic reaction in the conversion of urea to ammonia and carbon dioxide and Example 11-1 deals with the interconversion of D-glyceraldehyde 3-Phosphate and dihydroxyacetone phosphate. The Solver (EXAMPLEll-l.xls and EXAMPLEll-3.xls) uses the Michaehs-Menten (MM) formula to compute v i- The residual sums of squares between Vg(,j, and v j is then calculated. Using guessed values of and the Solver uses a search optimization technique to determine MM parameters. The values of and in Example 11-1 are ... 

Application of equation 10 to the experimental D vs. [HSOIJ] data determined at 25°C and both 1 and 2 M acidity yielded straight line plots with slopes indistinguishable from zero and reproduced the Bi values determined in a non-linear regression fit of the data. This result implies no adsorption of PuSO by the resin and justifies use of the simpler data treatment represented by equation 2. A similar analysis of the Th(IV)-HSOiJ system done by Zielen (9) likewise produced results consistent with no adsorption of ThS0 + by Dowex AG50X12 resin. 

A somewhat improved correlation is obtained by non-linear regression as ... 

Data given in Tables 1-6 clearly show a significant dependence of P2 and p4 on amine concentration, that is, at least one of the apparent rate constants kj contains a concentration factor. Thus, according to the mathematical considerations outlined in the Analysis of Data Paragraph, both p2, P4 exponents and the derived variables -(P2 + p)4> P2 P4 ind Z (see Eqns. 8-12) are the combinations of the apparent rate constants (kj). To characterize these dependences, derived variables -(p2+p)4, P2 P4 and Z (Eqns. 8,11 and 12) were correlated with the amine concentration using a non-linear regression program to find the best fit. Computation resulted in a linear dependence for -(p2 + p)4 and Z, that is... 

Kinetic analysis of the data obtained in differential reactors is straightforward. One may assume that rates arc directly measured for average concentrations between the inlet and the outlet composition. Kinetic analysis of the data produced in integral reactors is more difficult, as balance equations can rarely be solved analytically. The kinetic analysis requires numerical integration of balance equations in combination with non-linear regression techniques and thus it requires the use of computers. 

Model parameters estimated by linear regression, weighted linear regression, and unweighted non-linear regression are shown in Table B-1. 

Parameter Unweighted linear regression Weighted linear regression Non-linear regression... 

Note that the lipophilicity parameter log P is defined as a decimal logarithm. The parabolic equation is only non-linear in the variable log P, but is linear in the coefficients. Hence, it can be solved by multiple linear regression (see Section 10.8). The bilinear equation, however, is non-linear in both the variable P and the coefficients, and can only be solved by means of non-linear regression techniques (see Chapter 11). It is approximately linear with a positive slope (/ ,) for small values of log P, while it is also approximately linear with a negative slope b + b for large values of log P. The term bilinear is used in this context to indicate that the QSAR model can be resolved into two linear relations for small and for large values of P, respectively. This definition differs from the one which has been introduced in the context of principal components analysis in Chapter 17. 

The coefficients of the regression are all highly significant (p < 0.0001) and the fit of the model to the observed data is shown in Fig. 37.1. Using non-linear regression we obtain the bilinear Hansch model for the bactericidal activities of the 17 phenol analogs (Table 37.2) ... 

There have also been attempts to describe the temporal aspects of perception from first principles, the model including the effects of adaptation and integration of perceived stimuli. The parameters in the specific analytical model derived were estimated using non-linear regression [14]. Another recent development is to describe each individual TI-curve,/j(r), i = 1, 2,..., n, as derived from a prototype curve, S t). Each individual Tl-curve can be obtained from the prototype curve by shrinking or stretching the (horizontal) time axis and the (vertical) intensity axis, i.e. fff) = a, 5(b, t). The least squares fit is found in an iterative procedure, alternately adapting the parameter sets (a, Zi, for 1=1,2,..., n and the shape of the prototype curve [15],... 

Non-linear models, such as described by the Michaelis-Menten equation, can sometimes be linearized by a suitable transformation of the variables. In that case they are called intrinsically linear (Section 11.2.1) and are amenable to ordinary linear regression. This way, the use of non-linear regression can be obviated. As we have pointed out, the price for this convenience may have to be paid in the form of a serious violation of the requirement for homoscedasticity, in which case one must resort to non-parametric methods of regression (Section 12.1.5). 

B. Waiczack and D.L. Massart, The radial basis functions — partial least squares approach as a flexible non-linear regression technique. Anal. Chim. Acta, 331 (1996) 177-185. 

Hartley, H.O., "The Modified Gauss-Newton Method for the Fitting of Non-Linear Regression Functions by Least Squares", Technometrics, 3(2), 269-280(1961). 

The authors are indebted to Messrs. L.T. Hillegers and A.G. Swenker for assistance in the non-linear regression analysis and the parameter estimation, to Mr. J.H.M. Palmen for experimental help, to Mr. G. Schuler for drawing the figures and to Mrs. 

Nonlinearity is a subject the specifics of which are not prolifically or extensively discussed as a specific topic in the multivariate calibration literature, to say the least. Textbooks routinely cover the issues of multiple linear regression and nonlinearity, but do not cover the issue with full-spectrum methods such as PCR and PLS. Some discussion does exist relative to multiple linear regression, for example in Chemometrics A Textbook by D.L. Massart et al. [6], see Section 2.1, Linear Regression (pp. 167-175) and Section 2.2, Non-linear Regression, (pp. 175-181). The authors state,... 

Examples Polynomial regression is applied in problem Pi.03.02. Several examples of POLYMATH multilinear regression are in sections P3.06, P3.08 and P3.10. A non-linear regression is worked out in PI.02.07. 

Non linear regression would be needed to find the five constants. 

As predicted, the points next to the ends of the profile give unreliable (negative) estimates because the fluxes at the end points themselves become a substantial fraction of the fluxes in the neighborhood. In addition, noise in sections with rather flat gradients becomes a problem. More elaborated non-linear regression techniques should be used to handle this specific problem. Nevertheless, the profile between IS and SO/rm, where steep Ce variations are observed, shows evidence for substantial changes of the Off which seems to hint at much faster diffusion when this element is only in trace amounts in the apatite lattice, o... 

Opfermann, J., Kinetic analysis using multivariate non-linear regression. 

---