Linear PCA

A PCA system is called linear when all of the conditional probabilities P(5 Sj e A/i ) can be expressed as a sum of two-spin products [rujan87]  

Linear rules have the important property that all single-spin expectations and multi- spin correlations are decoupled. Calculating the expectation Sk t+ii for example, we see that 

Another consequence of linearity is that the mean-field approximation becomes exact. Prom equations 7.82 and 7.88, we can immediately see that the mean-field iterative equation 

---

Initially, the whole data set was analyzed by the linear PCA. By examining the behaviors of the process data in the projection spaces defined by small number of principal components, it... 

Non-linear PCA can be obtained in many different ways. Some methods make use of higher order terms of the data (e.g. squares, cross-products), non-linear transformations (e.g. logarithms), metrics that differ from the usual Euclidean one (e.g. city-block distance) or specialized applications of neural networks [50]. The objective of these methods is to increase the amount of variance in the data that is explained by the first two or three components of the analysis. We only provide a brief outline of the various approaches, with the exception of neural networks for which the reader is referred to Chapter 44. 

One approach is to extend the columns of a measurement table by means of their powers and cross-products. An example of such non-linear PCA is discussed in Section 37.2.1 in an application of QSAR, where biological activity was known to be related to the hydrophobic constant by means of a quadratic function. In this case it made sense to add the square of a particular column to the original measurement table. This procedure, however, tends to increase the redundancy in the data. 

The logarithmic transformation prior to column- or double-centered PCA (Section 31.3) can be considered as a special case of non-linear PCA. The procedure tends to make the row- and column-variances more homogeneous, and allows us to interpret the resulting biplots in terms of log ratios. 

The theory of the non-linear PCA biplot has been developed by Gower [49] and can be described as follows. We first assume that a column-centered measurement table X is decomposed by means of classical (or linear) PCA into a matrix of factor scores S and a matrix of factor loadings L ... 

Fig. 31.17. (a) In a classical PCA biplot, data values xy can be estimated by means of perpendicular projection of the ith row-point upon a unipolar axis which represents theyth column-item of the data table X. In this case the axis is a straight line through the origin (represented by a small cross), (b) In a non-linear PCA biplot, the jth column-item traces out a curvilinear trajectory. The data value is now estimated by defining the shortest distance between the ith row point and theyth trajectory. 

Non linear PCA algorithms have also been developed to provide a representation along principle curves rather than principal directions... 

The 0-NLPCA network has 8-6-10-12 neurons in each layer, yielding a prototype model with 6 principal components (PCs). For comparison, the linear PCA was also applied to the same data. As a performance criterion, the root mean square of error (RMSE) was evaluated to compare the prediction ability of the developed PCA and O-NLPCA models on the training and validation data. While the linear PCA gave 0.3021 and 0.3227 RMSE on training and validation data sets, respectively, the O-NLPCA provided 0.2526 and 0.2244 RMSE. This suggests that to capture the same amount of information, the linear PCA entails utilization of more principal components than its nonlinear counterpart. As a result, the information embedded in the nonlinear principal components addresses the underlying events more efficiently than the linear ones. 

Scholz,M.,Kaplan,E,Guy,C.L.,Kopka,X,Selbig,X(2005) Non-linear PCA a missing data approach. Bioinformatics, 21, 3887-3895. 

Successive PCA and Wavelet analysis processes improve small flaw detection (figure 14), because small size involves linear physical processes, where PCA is efficient. 

We have to apply projection techniques which allow us to plot the hyperspaces onto two- or three-dimensional space. Principal Component Analysis (PCA) is a method that is fit for performing this task it is described in Section 9.4.4. PCA operates with latent variables, which are linear combinations of the original variables. 

Kohonen network Conceptual clustering Principal Component Analysis (PCA) Decision trees Partial Least Squares (PLS) Multiple Linear Regression (MLR) Counter-propagation networks Back-propagation networks Genetic algorithms (GA)... 

Sections 9A.2-9A.6 introduce different multivariate data analysis methods, including Multiple Linear Regression (MLR), Principal Component Analysis (PCA), Principal Component Regression (PCR) and Partial Least Squares regression (PLS). 

PLS is a linear regression extension of PCA which is used to connect the information in two blocks of variables X and Yto each other. It can be applied even if the features are highly correlated. 

The previously mentioned data set with a total of 115 compounds has already been studied by other statistical methods such as Principal Component Analysis (PCA), Linear Discriminant Analysis, and the Partial Least Squares (PLS) method [39]. Thus, the choice and selection of descriptors has already been accomplished. 

If, as in this case, all, or nearly all, of the spectral variance is linearly correlated to the concentration variance, the optimum PLS factors, W, and the corresponding PLS spectral factors, P, will tend to be very similar to each other. And W and P will, in turn, tend to be very similar to the PCA spectral factors. If, on the other hand, there is a significant amount of spectral variance that is... 

Principal Component Analysis (PCA). Principal component analysis is an extremely important method within the area of chemometrics. By this type of mathematical treatment one finds the main variation in a multidimensional data set by creating new linear combinations of the raw data (e.g. spectral variables) [4]. The method is superior when dealing with highly collinear variables as is the case in most spectroscopic techniques two neighbor wavelengths show almost the same variation. 

A first introduction to principal components analysis (PCA) has been given in Chapter 17. Here, we present the method from a more general point of view, which encompasses several variants of PCA. Basically, all these variants have in common that they produce linear combinations of the original columns in a measurement table. These linear combinations represent a kind of abstract measurements or factors that are better descriptors for structure or pattern in the data than the original measurements [1]. The former are also referred to as latent variables [2], while the latter are called manifest variables. Often one finds that a few of these abstract measurements account for a large proportion of the variation in the data. In that case one can study structure and pattern in a reduced space which is possibly two- or three-dimensional. 

In order to apply RBL or GRAFA successfully some attention has to be paid to the quality of the data. Like any other multivariate technique, the results obtained by RBL and GRAFA are affected by non-linearity of the data and heteroscedast-icity of the noise. By both phenomena the rank of the data matrix is higher than the number of species present in the sample. This has been demonstrated on the PCA results obtained for an anthracene standard solution eluted and detected by three different brands of diode array detectors [37]. In all three cases significant second eigenvalues were obtained and structure is seen in the second principal component. 

The NMR spectra using PCA and Linear Discriminant Analysis (LDA) obtained for instant spray dried coffees from a number of different manufacturers demonstrated [8] that the concentration of the extracted molecules is generally high enough for clear detection. The compound 5-(hydroxymethy)-2-furaldehyde was identified as the primary marker of differentiation between two groups of coffees. This method may be used to determine whether a fraudulent retailer is selling an inferior quality product marked as being from a reputable manufacturer [8]. 

Techniques for multivariate input analysis reduce the data dimensionality by projecting the variables on a linear or nonlinear hypersurface and then describe the input data with a smaller number of attributes of the hypersurface. Among the most popular methods based on linear projection is principal component analysis (PCA). Those based on nonlinear projection are nonlinear PCA (NLPCA) and clustering methods. 

Here xik is an estimated value of a variable at a given point in time. Given that the estimate is calculated based on a model of variability, i.e., PCA, then Qi can reflect error relative to principal components for known data. A given pattern of data, x, can be classified based on a threshold value of Qi determined from analyzing the variability of the known data patterns. In this way, the -statistic will detect changes that violate the model used to estimate x. The 0-statistic threshold for methods based on linear projection such as PCA and PLS for Gaussian distributed data can be determined from the eigenvalues of the components not included in the model (Jack-son, 1992). 

Now, what is interesting about this situation is that ordinary regression theory and the theory of PCA and PLS specify that the model generated must be linear in the coefficients. Nothing is specified about the nature of the data (except that it be noise-free, as our simulated data is) the data may be non-linear to any degree. Ordinarily this is not a problem because any data transform may be used to linearize the data, if that is desirable. 

Table 33-1 Summary of results obtained from synthetic linearity data using one PCA or PLS factor. We present only those performance results listed by the data analyst as Correlation Coefficient and Standard Error of Estimate...

Principal component analysis (PCA) of the soil physico-chemical or the antibiotic resistance data set was performed with the SPSS software. Before PCA, the row MPN values were log-ratio transformed (ter Braak and Smilauer 1998) each MPN was logio -transformed, then, divided by sum of the 16 log-transformed values. Simple linear regression analysis between scores on PCs based on the antibiotic resistance profiles and the soil physico-chemical characteristics was also performed using the SPSS software. To find the PCs that significantly explain variation of SFI or SEF value, multiple regression analysis between SFI or SEF values and PC scores was also performed using the SPSS software. The stepwise method at the default criteria (p=0.05 for inclusion and 0.10 for removal) was chosen. 

---