Variance maximum

Even though two factors are all we need to span this data, we could find as many factors as there are wavelengths in the spectra. Each successive factor is identical to each successive eigenvector of the data. Each successive factor will capture the maximum variance of the data that was not yet spanned by the earlier factors. Each successive factor must be mutual orthogonal to all the factors that precede it. Let s continue on and plot the third factor for this data set. The plots are shown in Figures 37 and 38. 

Rotated axes are characterized by their position in the original space, given by the vectors = [cos0 -sin0] and F = [sin0 cos0] (see Fig. 34.8). In PCA or FA, these axes fulfil specific constraints (see Chapter 17). For instance, in PCA the direction of k is the direction of the maximum variance of all points projected on this axis. A possible constraint in FA is maximum simplicity of k, which is explained in Section 34.2.3. The new axes (k,l) define another basis of the same space. The position of the vector [x,- y,] is now [fc, /,] relative to these axes. 

Principal component regression Linear projection Fixed shape, linear a, maximum variance of projected inputs (3, minimum output prediction error... 

The PCA can be interpreted geometrically by rotation of the m-dimensional coordinate system of the original variables into a new coordinate system of principal components. The new axes are stretched in such a way that the first principal component pi is extended in direction of the maximum variance of the data, p2 orthogonal to pi in direction of the remaining maximum variance etc. In Fig. 8.15 a schematic example is presented that shows the reduction of the three dimensions of the original data into two principal components. 

For principal component analysis (PCA), the criterion is maximum variance of the scores, providing an optimal representation of the Euclidean distances between the objects. 

Exploratory data analysis has the aim to learn about the data distribution (clusters, groups of similar objects). In multivariate data analysis, an X-matrix (objects/samples characterized by a set of variables/measurements) is considered. Most used method for this purpose is PCA, which uses latent variables with maximum variance of the scores (Chapter 3). Another approach is cluster analysis (Chapter 6). 

The direction in a variable space that best preserves the relative distances between the objects is a latent variable which has maximum variance of the scores (these are the projected data values on the latent variable). This direction is called by definition the first principal component (PCI). It is defined by a loading vector... 

PCA is sensitive with respect to outliers. Outliers are unduly increasing classical measures of variance (that means nonrobust measures), and since the PCs are following directions of maximum variance, they will be attracted by outliers. Figure 3.8 (left) shows this effect for classical PCA. In Figure 3.8 (right), a robust version of PCA was taken (the method is described in Section 3.5). The PCs are defined as directions maximizing A robust measure of variance (see Section 2.3) which is not inflated by the outlier group. As a result, the PCs explain the variability of the nonoutliers which refer to the reliable data information. 

FIGURE 4.12 Linear latent variable with maximum variance of scores (PCA) and maximum correlation coefficient between y and scores (OLS). Scatter plot of a demo data set with 10 objects and two variables (x1 x2, mean-centered) the diameter of the symbols is proportional to a property y R2 denotes the squared correlation coefficients between y and x1 y and x2, y and PCI scores, y and y from OLS. 

First PLS-component is calculated as the latent variable which has maximum covariance between the scores and modeled property y. Note that the criterion covariance is a compromise between maximum correlation coefficient (OLS) and maximum variance (PCA). 

FIGURE 6.2 Representation of multivariate data by icons, faces, and music for human cluster analysis and classification in a demo example with mass spectra. Mass spectra have first been transformed by modulo-14 summation (see Section 7.4.4) and from the resulting 14 variables, 8 variables with maximum variance have been selected and scaled to integer values between 1 and 5. A, typical pattern for aromatic hydrocarbons B, typical pattern for alkanes C, typical pattern for alkenes 1 and 2, unknowns (2-methyl-heptane and meta-xylene). The 5x8 data matrix has been used to draw faces (by function faces in the R-library Tea-chingDemos ), segment icons (by R-function stars ), and to create small melodies (Varmuza 1986). Both unknowns can be easily assigned to the correct class by all three representations. 

Note The used variable sets are 14 modulo-14 features (autoscaled) 2 and 3 PCA scores calculated from the autoscaled modulo-14 features peak intensities at 14 selected mass numbers (with maximum variances of the peak intensities) 50 mass spectral features. The numbers of correct predictions are from a leave-one-out test n is the number of spectra in the five DBE groups... 

An often-overlooked issue is the inherent non-orthogonality of coordinate systems used to portray data points. Almost universally a Euclidean coordinate system is used. This assumes that the original variables are orthogonal, that is, are uncorrelated, when it is well known that this is generally not the case. Typically, principal component analysis (PCA) is performed to generate a putative orthogonal coordinate system each of whose axes correspond to directions of maximum variance in the transformed space. This, however, is not quite cor-... 

Usual procedures for the selection of the common best basis are based on maximum variance criteria (Walczak and Massart, 2000). For instance, the variance spectrum procedure computes at first the variance of all the variables and arranges them into a vector, which has the significance of a spectrum of the variance. The wavelet decomposition is applied onto this vector and the best basis obtained is used to transform and to compress all the objects. Instead, the variance tree procedure applies the wavelet decomposition to all of the objects, obtaining a wavelet tree for each of them. Then, the variance of each coefficient, approximation or detail, is computed, and the variance values are structured into a tree of variances. The best basis derived from this tree is used to transform and to compress all the objects. 

Table Al Critical values of Cochran s maximum variance rati(f ...

The method of PLS bears some relation to principal component analysis instead of Lnding the hyperplanes of maximum variance, it Lnds a linear model describing some predicted variables in terms of other observable variables. It is used to Lnd the fundamental relations between two matrices (X andY), that is, a latent variable approach to modeling the covariance structures in these two spaces. A PLS model will try to Lnd the multidimensional direction irMIspace that explains the maximum multidimensional variance direction in flrfspace. 

In order to handle multiple Y-variables, an extension of the PLS regression method discussed earlier, called PLS-2, must be used.1 The algorithm for the PLS-2 method is quite similar to the PLS algorithms discussed earlier. Just like the PLS method, this method determines each compressed variable (latent variable) based on the maximum variance explained in both X and Y. The only difference is that Y is now a matrix that contains several Y-variables. For PLS-2, the second equation in the PLS model (Equation 8.36) can be replaced with the following ... 

This method is applicable when data are to be inspected and characterized. PCA is easily understood by graphical illustrations, for example, by a two-dimensional co-ordinate system with a number of points in it (Figure 6.25). The first principal component (PC) is the line with the closest fit to these points [12]. Unless the point swarm has, for example, the shape of a circle, the position of the first PC is unambiguous. Because the first PC is the line of closest fit, it is also the line that explains most of the variation (maximum variance) in the data [13]. Therefore it is called the principal component. 

The most important among the known criteria of design optimality is the requirement of D- and G-optimality. A design is said to be D-optimal when it minimizes the volume of the scatter ellipsoid for estimates of regression equation coefficients. The property of G-optimality provides the least maximum variance of predicted response values in a region under investigation. 

The first principal component explains the maximum variance (information) in the data set. Subsequent components describe the maximum part of the remaining variance subject to the condition that ... 

The first LV accounts for the maximum variance in the descriptor set, and has the highest correlation with the dependent variable. 

Figure 3.4 shows the values of the variance for all angles from 0 to 360 degrees (computed in steps of 1 degree). Maximum variance is at an angle of 27 degrees ... 

Fig. 3.4 Variance of data set A during rotation of the coordinate system (in polar coordinates). Maximum variance vpci is obtained for the direction PCI (at an angle of 27 degrees). vxi and vx2 are" the variances of the original features ai and X2. PCI and PC2 are the two principal components of the data.

A principal component analysis is reasonable only when the intrinsic dimensionality is much smaller than the dimensionality of the original data. This is the case for features related by high absolute values of the correlation coefficients. Whenever correlation between features is small, a significant direction of maximum variance cannot be found (Fig. 3.7) all principal components participate in the description of the data structure hence a reduction of data by principal component analysis is not possible. 

---