Unit variance scaling

The two main ways of data pre-processing are mean-centering and scaling. Mean-centering is a procedure by which one computes the means for each column (variable), and then subtracts them from each element of the column. One can do the same with the rows (i.e., for each object). ScaUng is a a slightly more sophisticated procedure. Let us consider unit-variance scaling. First we calculate the standard deviation of each column, and then we divide each element of the column by the deviation. 

Variance scaling is performed on a variable by variable basis. In other words, we would variance scale a set the concentration values of a data set on a component by component basis. Starting with the first component, we compute the total variance of the concentrations of that component. There are several variations on variance scaling. First, we will consider the most the method which adjusts all the variables to exactly unit variance. To do this we compute the variance of the variable, and then use the variance to scale all the concentrations of all the samples so that the new variance for the component is equal to unity. 

Figure C3. The data from Figure Cl after scaling to unit variance.

They change Px into Py, but the difference is so minor that they are often considered the same distribution, and are denoted by the same name. The transformation can be used to transform the distribution to a standard form, for instance one with zero average and unit variance. In the case of lattice distributions one employs (5.4) to make the lattice points coincide with integers. In fact, the use of (5.4) to reduce the distribution to a simple form is often done tacitly, or in the guise of choosing the zero and the unit on the scale. 

Both PCA and PLS are sensitive to the scaling of the variables. It is therefore customary to scale the data variable-wise to zero mean and unit variance. However, the possibility to employ a different scaling whenever this seems to be appropriate should be kept in mind. For instance, a well-known important variable may be given a larger variance, for example, 3.0 instead of 1.0. Another situation in which a different scaling may be considered is when the variables are obviously blocked in such a way that each block contains closely related variables. Each block may then be given equal variance whereby the blocks have equal chances to influence the direction of the PC or PLS dimension. 

The variances and covariance of the descriptors are given by the matrix (X — X) (X — X), in which the diagonal elements are the variances of the variables and the off-diagonal elements are the covariances. When the data have been scaled to unit variance, this matrix is called the correlation matrix and the off-diagonal elements are correlation coefficients for the correlations between the variables, and the sum of the variances is equal to the number of variables. 

X and Y denote the scaled and mean centered matrices of the variations in the X and Y space, respectively. Scaling to unit variance is usually employed. PLS modelling involves the factorization of the matrices X and Y into matrices of scores and loadings. In matrix notation, the model... 

There are certain situations where scaling to unit variance is not the preferred procedure and where no scaling at all is better. If all descriptor variables are of the same kind and measured in the same unit, e.g. intensities of spectral absorbtion at different frequencies or peak heights in chromatographic profiles, it is sometimes unnecessary to scale the variables. Autoscaling such variables would exaggerate minor variations. Another case, when scaling may be unnecessary is when a variable... 

If the values of some descriptor vary in magnitudes over the set of compounds it is difficult to assume that a linear model will be a good approximation to account for such large variations. In these cases, a better model can often be obtained after a logarithmic transformation of this variable prior to scaling to unit variance. 

When the descriptors have been scaled to unit variance, the matrix (X - X) (X — X) is the correlation matrix, and the sum of squares will be equal to K (the number of descriptors). 

The procedure by which the factorization described above is carried out on a data matrix obtained after centering and scaling each descriptor to unit variance is often called Factor Analysis, (The term "Factor analysis" in mathematical statistics has a slightly different meaning which will not be discussed here. For details of this, see [9]). It can be effected by computing the eigenvectors to (X - X) (X - X). Another, and more efficient method is to use a procedure called Singular value descomposition (SVD), see Appendix 15B. 

A standard procedure is to scale each descriptor to unit variance. 

Prior to computing the PLS model, the blocks were centred by subtracting the averages, and the variables were scaled to unit variance over the set of experiments. For the scaled and mean centred x variables, the following notation is used The scaled variable from Xj is denoted Xjj the scaled variable from the squared variable is denoted x the scaled variable from the cross-product XjXj is denoted Xjj. For the scaled response variable, y the symbol is used. 

PCA is a least square method and therefore its results depend on data scaling. The initial variance of a column variable partly determines its importance in the model. In order to avoid the problem of over- or under-representation of variables, column variables are scaled to unit variance before analysis. The column average is then subtracted from each variable, which, from a statistical point of view, corresponds to moving the multivariate system to the center of the data, which becomes the starting point of the mathematical analysis. The same auto-scaUng and centering procedures are applied in PLS discriminant analysis. 

Assume that p sensors are monitored. The calibration data set is collected for n time steps and the measurements are stored in the block matrix X after being scaled to unit-variance and zero-mean columns. x(A ) is the p X 1 vector of observations at the fcth sampling time. The transpose of x k) is the fcth row of X. 

After scaling, each observation has unit variance but different means. Standardization involves both centering and scaling. Each observations has the mean subtracted and is then divided by the column standard deviation,... 

After standardization, each observation has zero mean and unit variance. Typically, centering is tried first because the estimates and the units of the parameter associated with x remain the same, whereas with scaling or standardization the units and the parameter estimates themselves are different. If centering fails, usually standardization is tried next. 

---