Regression transformations

It may look weird to treat the Singular Value Decomposition SVD technique as a tool for data transformation, simply because SVD is the same as PCA. However, if we recall how PCR (Principal Component Regression) works, then we are really allowed to handle SVD in the way mentioned above. Indeed, what we do with PCR is, first of all, to transform the initial data matrix X in the way described by Eqs. (10) and (11). 

On the other hand, techniques like Principle Component Analysis (PCA) or Partial Least Squares Regression (PLS) (see Section 9.4.6) are used for transforming the descriptor set into smaller sets with higher information density. The disadvantage of such methods is that the transformed descriptors may not be directly related to single physical effects or structural features, and the derived models are thus less interpretable. 

The following paper discusses some of the problems that maybe encountered when using linear regression to model data that have been mathematically transformed into a linear form. 

Numeric-to-numeric transformations are used as empirical mathematical models where the adaptive characteristics of neural networks learn to map between numeric sets of input-output data. In these modehng apphcations, neural networks are used as an alternative to traditional data regression schemes based on regression of plant data. Backpropagation networks have been widely used for this purpose. 

Figure 4.49 Linear regression for the 2-parameter Weibull transformed load frequency data...

Figure 4.60 Linear regression for the Normal distribution transformed hardness data...

Thus, Tis a linear function of the new independent variables, X, X2,. Linear regression analysis is used to ht linear models to experimental data. The case of three independent variables will be used for illustrative purposes, although there can be any number of independent variables provided the model remains linear. The dependent variable Y can be directly measured or it can be a mathematical transformation of a directly measured variable. If transformed variables are used, the htting procedure minimizes the sum-of-squares for the differences... 

The various independent variables can be the actual experimental variables or transformations of them. Dilferent transformations can be used for different variables. The independent variables need not be actually independent. For example, linear regression analysis can be used to fit a cubic equation by setting X, X and Z as the independent variables. 

However, it can be shown easily that a regression line of a given set of points does not remain the regression line after the transformation, and also that the correlation coefficient is altered. Let us denote in the original coordinates log k2 versus logki r, the correlation coefficient b2.i and l/bi.2 the slopes of... 

One can further compute the slopes b2.i and l/bi.2 of the real regression lines, drawn in the log k2 versus log ki plane and transformed into the E versus log A plane ... 

However, it is not proper to apply the regression analysis in the coordinates AH versus AS or AS versus AG , nor to draw lines in these coordinates. The reasons are the same as in Sec. IV.B., and the problem can likewise be treated as a coordinate transformation. Let us denote rcH as the correlation coefficient in the original (statistically correct) coordinates AH versus AG , in which sq and sh are the standard deviations of the two variables from their averages. After transformation to the coordinates TAS versus AG or AH versus TAS , the new correlation coefficients ros and rsH. respectively, are given by the following equations. (The constant T is without effect on the correlation coefficient.)... 

It can further be shown how the slopes of regression lines are changed during the transformation. Let shTch/sg be the slope of the real... 

Both aspects are combined in Fig. (2.20) and Table 2.16, where the linear coordinates are x resp. y, the logarithmic ones , resp. v. Regression coefficients established for the lin/lin plot are a, b, whereas those for the transformed coordinates are p, q. 

LEGEND NN linear regression without data transformation LL idem, using logarithmically transformed axes not interpretable... 

To benchmark our learning methodology with alternative conventional approaches, we used the same 500 (x, y) data records and followed the usual regression analysis steps (including stepwise variable selection, examination of residuals, and variable transformations) to find an approximate empirical model, / (x), with a coefficient of determination = 0.79. This model is given by... 

Because of peak overlappings in the first- and second-derivative spectra, conventional spectrophotometry cannot be applied satisfactorily for quantitative analysis, and the interpretation cannot be resolved by the zero-crossing technique. A chemometric approach improves precision and predictability, e.g., by the application of classical least sqnares (CLS), principal component regression (PCR), partial least squares (PLS), and iterative target transformation factor analysis (ITTFA), appropriate interpretations were found from the direct and first- and second-derivative absorption spectra. When five colorant combinations of sixteen mixtures of colorants from commercial food products were evaluated, the results were compared by the application of different chemometric approaches. The ITTFA analysis offered better precision than CLS, PCR, and PLS, and calibrations based on first-derivative data provided some advantages for all four methods. ... 

The plot of pH against titrant volume added is called a potentiometric titration curve. The latter curve is usually transformed into a Bjerrum plot [8, 24, 27], for better visual indication of overlapping pKiS or for pffjS below 3 or above 10. The actual values of pKa are determined by weighted nonlinear regression analysis [25-27]. 

It must be emphasized that Procrustes analysis is not a regression technique. It only involves the allowed operations of translation, rotation and reflection which preserve distances between objects. Regression allows any linear transformation there is no normality or orthogonality restriction to the columns of the matrix B transforming X. Because such restrictions are released in a regression setting Y = XB will fit Y more closely than the Procrustes match Y = XR (see Section 35.3). 

Non-linear models, such as described by the Michaelis-Menten equation, can sometimes be linearized by a suitable transformation of the variables. In that case they are called intrinsically linear (Section 11.2.1) and are amenable to ordinary linear regression. This way, the use of non-linear regression can be obviated. As we have pointed out, the price for this convenience may have to be paid in the form of a serious violation of the requirement for homoscedasticity, in which case one must resort to non-parametric methods of regression (Section 12.1.5). 

Having the smoothed values of the state variables at each sampling point and the derivatives, q we have essentially transformed the problem to a "usual" linear regression problem. The parameter vector is obtained by minimizing the following LS objective function... 

---