Data matrices standardization

In order for a solution for the systems of equations expressed in equation 11 to exist, the number of sensors must be at least equal to the number of analytes. To proceed, the analyst must first determine the sensitivity factors using external standards, ie, solve equation 11 for Kusing known C and R. Because concentration C is generally not a square data matrix, equation 11 is solved by the generalized inverse method. K is given by... 

To compute the variance, we first find the mean concentration for that component over all of the samples. We then subtract this mean value from the concentration value of this component for each sample and square this difference. We then sum all of these squares and divide by the degrees of freedom (number of samples minus 1). The square root of the variance is the standard deviation. We adjust the variance to unity by dividing the concentration value of this component for each sample by the standard deviation. Finally, if we do not wish mean-centered data, we add back the mean concentrations that were initially subtracted. Equations [Cl] and [C2] show this procedure algebraically for component, k, held in a column-wise data matrix. 

The final choice will have to be made during method development and/or analysis of the real samples, e.g. one of the ions selected may provide superb data from standard solutions but show a high matrix background on all or, perversely, on only a small number of samples, which will preclude its/their use. 

To further analyze the relationships within descriptor space we performed a principle component analysis of the whole data matrix. Descriptors have been normalized before the analysis to have a mean of 0 and standard deviation of 1. The first two principal components explain 78% of variance within the data. The resultant loadings, which characterize contributions of the original descriptors to these principal components, are shown on Fig. 5.8. On the plot we can see that PSA, Hhed and Uhba are indeed closely grouped together. Calculated octanol-water partition coefficient CLOGP is located in the opposite corner of the property space. This analysis also demonstrates that CLOGP and PSA are the two parameters with... 

In 1978, Ho et al. [33] published an algorithm for rank annihilation factor analysis. The procedure requires two bilinear data sets, a calibration standard set Xj and a sample set X . The calibration set is obtained by measuring a standard mixture which contains known amounts of the analytes of interest. The sample set contains the measurements of the sample in which the analytes have to be quantified. Let us assume that we are only interested in one analyte. By a PCA we obtain the rank R of the data matrix X which is theoretically equal to 1 + n, where rt is the number of interfering compounds. Because the calibration set contains only one compound, its rank R is equal to one. 

In order to apply residual bilinearization [35] at least two data sets are needed X which is the data set measured for the unknown sample and X which is the data matrix of a calibration standard, containing the analyte of interest. In the absence of interferences these two data matrices are related to each other as follows ... 

In order to apply RBL or GRAFA successfully some attention has to be paid to the quality of the data. Like any other multivariate technique, the results obtained by RBL and GRAFA are affected by non-linearity of the data and heteroscedast-icity of the noise. By both phenomena the rank of the data matrix is higher than the number of species present in the sample. This has been demonstrated on the PCA results obtained for an anthracene standard solution eluted and detected by three different brands of diode array detectors [37]. In all three cases significant second eigenvalues were obtained and structure is seen in the second principal component. 

The basis of all data-analytical procedures is the data matrix (Eq. 8.10). In many cases the original data x j have to be transformed, either into standardized data ... 

The raw data in the more comprehensive study (61) were conversions, determined in duplicate, when each of 104 coals selected from three geological provinces was heated with tetralin under standard conditions, together with the results of 14 commonly made analytical determinations for each coal. An early observation in this study was that when data for all 104 samples were plotted against volatile matter, a steady decrease of conversion with decreasing volatile matter was apparent. But there was a great deal of scatter (r=0.85). In any case, the formal requirements that make possible the employment of valid statistical analyses were not met by the data matrix, as evidenced by skewed and bimodal relationships between the variables the sample set was heterogeneous. ... 

D-NMR methods are highly useful for structure elucidation. Jeener described the principles of the first 2D-NMR experiment in 1971 [31]. In standard NMR nomenclature, a data set is referred to by one, i.e., less than the total number of actual dimensions, since the intensity dimension is implied. The 2D-data matrix therefore can be described as a plot containing two frequency dimensions. The inherent third dimension is the intensity of the correlations within the data matrix. This is the case in ID NMR data as well. The implied second dimension actually reflects the intensity of the peaks of a certain resonance... 

If, however, the standard deviations, ayij, for all elements of the matrix Y are known or can be estimated reliably, it does make sense to use this information in the data analysis. Then, instead of the sum of squares, it is the sum of all appropriately weighted and squared residuals that has to be minimised. This is known as the chi-square or x2 -fitting. If the data matrix Y has the dimensions nsxnl, %2 is defined by... 

Standard deviation of data matrix unexplained by PC model. 

A kind of logarithmic transform, such as In (1 -I- x), is used in spectral maps within row and column centring and global standardization (division by the standard deviation around the mean of all the values of the data matrix). 

Cross-Validation. Cross-validation is a comparison of validation parameters when two or more bioanalytical methods are used to generate data within the same study or across different studies. An example of cross-validation would be a situation where an original validated bioanalytical method serves as the reference and the revised bioanalytical method is the comparator. The comparisons should be done both ways. When sample analyses within a single study are conducted at more than one site or more than one laboratory, cross-validation with spiked matrix standards and subject samples should be conducted at each site or laboratory... 

Compare sample results against expectations and compare control sample results against the acceptable range (see Critical Parameters). Evaluate replicate results and recoveries for acceptability. Expectations are based upon historical data with a specific matrix, standard references, or expected results (e.g., claims). Acceptable ranges are determined during method validation. 

This empirical statistical function, based on the residual standard deviation (RSD), reaches a minimum when the correct number of factors are chosen. It allows one to reduce the number of columns of R from L to K eigenvectors or pure components. These K independent and orthogonal eigenvectors are sufficient to reproduce the original data matrix. As they are the result of a mathematical treatment of matrices, they have no physical meaning. A transformation (i.e. a rotation of the eigenvectors space) is required to find other equivalent eigenvectors which correspond to pure components. 

The simultaneous standardization of features and objects is the optimum for this example (Fig. 5-4). The minimum value of the whole data matrix is set to zero and the maximum value of the whole matrix is set to one. The original structures are more or less maintained. 

For demonstration we use the data of a cooperative test [DOERFFEL and ZWANZIGER, 1987], In this interlaboratory comparison five laboratories were involved in the analysis of slag samples three times for seven chemical elements. So the (15, 7)-data matrix consists of 5 times 3 rows and 7 columns. The raw data in % are given in Tab. 5-2. The data have been preprocessed by standardization (autoscaling). 

The function in Example 4.4 can be used to autoscale a data matrix. The function determines the size of the argument, its mean vector, and its standard deviation vector. On the last fine, a MATLAB programming trick is used to extend the mean vector and standard deviation vector into matrices having the same number of rows as the original argument prior to subtraction and division. The expression ones < r, i) creates an r x 1 column vector of ones. When used as an index in the statement mn (ones(r,1), ), it instructs MATLAB to replicate the mean vector r times to give a matrix having the dimensions r x c. 

In Equation 4.38, n and m are the numbers of rows and columns in the original data matrix, N is a proportionality constant, and a is the standard deviation of the error in the original data matrix. Malinowski proposed calculation of so-called reduced error eigenvalues, which are directly proportional to the square of the measurement error, cr. 

Standardise this matrix, and explain why this transformation is important. Why is it normal to use the population rather than the sample standard deviation All calculations below should be performed on this standardised data matrix. 

Why is the standard deviation a good measure of variable significance Reduce the dataset to 100 significant variables with the highest standard deviations to give a 10 x 100 data matrix. 

In chemometrics it is usual to use the population and not the sample standard deviation for standardising a data matrix. The reason is that we are not trying to estimate parameters in this case, but just to put different variables on a similar scale. 

---