Multivariate data preprocessing

The development of anal3dical techniques involves implementation of new and advanced computational methods that enable multivariate data preprocessing and their exploration. The metabolic profiles, previously determined using an adequate... 

E. de Noord, The influence of data preprocessing on the robustness nd parsimony of multivariate calibration models. Chemom. Intell. Lab. Systems, 23 (1994) 65-70,... 

In Chapter 2, we approach multivariate data analysis. This chapter will be helpful for getting familiar with the matrix notation used throughout the book. The art of statistical data analysis starts with an appropriate data preprocessing, and Section 2.2 mentions some basic transformation methods. The multivariate data information is contained in the covariance and distance matrix, respectively. Therefore, Sections... 

The data processing can be divided into three phases. Phase 1 is the removal of poor quality spectra with an automated routine. Phase 2 is the data preprocessing of the spectra, which passed the quality test. This usually entails some type of baseline correction and normalization process. Phase 3 is multivariate image reconstruction where the spectra are classified and reproduced as color points... 

Phase 2 - data preprocessing. There are many ways to process spectral data prior to multivariate image reconstruction and there is no ideal method that can be generally applied to all types of tissue. It is usual practice to correct the baseline to account for nonspecific matrix absorptions and scattering induced by the physical or bulk properties of the dehydrated tissue. One possible procedure is to fit a polynomial function to a preselected set of minima points and zero the baseline to these minima points. However, this type of fit can introduce artifacts because baseline variation can be so extreme that one set of baseline points may not account for all types of baseline variation. A more acceptable way to correct spectral baselines is to use the derivatives of the spectra. This can only be achieved if the S/N of the individual spectra is high and if an appropriate smoothing factor is introduced to reduce noise in the derivatized spectra. Derivatives serve two purposes they minimize broad... 

Data preprocessing is important in multivariate calibration. Indeed, the relationship between even basic procedures such as centring the columns is not always clear, most investigators following conventional methods, that have been developed for some popular application but are not always appropriately transferable. Variable selection and standardisation can have a significant influence on the performance of calibration models. 

Multivariate methods of mathematical analysis are needed that are robust, non-subjective and make use of all spectral information. A summary of reports using multivariate methods for the analysis of biomedical MRS data is listed in Table 1. In these studies, however, the requirements for a large patient cohort, data preprocessing/reduction and/or validation of classifiers are not met. 

O.E. Denoord, The Influence of Data Preprocessing on the Robustness and Parsimony of Multivariate Calibration Models, Chemometrics and Intelligent Laboratory Systems, 23(1) (1994), 65-70. 

Two of the most commonly employed methods of preprocessing multivariate data are mean centering and variance scaling of the spectra. Taken together, the application of mean centering and variance scaling is autoscaling. 

We have previously underlined that the absence of a common linear intensity axis (spectra not properly normalized) would lead to erroneous predictions in quantitative spectral analyses and in meaningless interpretations for exploratory data analyses. Another prerequisite for multivariate data analysis is that the data conform to the selected model. An assumption that applies to most of the multivariate methods is that the data are low rank bilinear. For most multivariate methods, this implies that the spectral axis must remain constant, that is, the signal(s) for a given chemical compound must appear at the same position in all the spectra. We will see how a different interval-based approach, not aimed at building chemometric models, but rather at spectral data preprocessing can effectively contribute to achieve an efficient and comprehensive horizontal signal alignment. 

Data preprocessing is an important component of successfully implementing multivariate analyses however, how best to preprocess fluorescence data sets is frequently a point of confusion. Both the type and order of preprocessing steps, and whether these are applied to rows (samples) or columns (variables), can affect the results (Bro and Smilde, 2003). 

As we have mentioned, the particular characterization task considered in this work is to determine attenuation in composite materials. At our hand we have a data acquisition system that can provide us with data from both PE and TT testing. The approach is to treat the attenuation problem as a multivariable regression problem where our target values, y , are the measured attenuation values (at different locations n) and where our input data are the (preprocessed) PE data vectors, u . The problem is to find a function iy = /(ii ), such that i), za jy, based on measured data, the so called training data. 

Usually, the raw data in a matrix are preprocessed before being submitted to multivariate analysis. A common operation is reduction by the mean or centering. Centering is a standard transformation of the data which is applied in principal components analysis (Section 31.3). Subtraction of the column-means from the elements in the corresponding columns of an nxp matrix X produces the matrix of... 

GLS preprocessing can be considered a more elaborate form of variable scaling, where, instead of each variable having its own scaling factor (as in autoscaling and variable-specific scaling), the variables are scaled to de-emphasize multivariate directions that are known to correspond to irrelevant spectral effects. Of course, the effectiveness of GLS depends on the ability to collect data that can be used to determine the difference effects, the accuracy of the measured difference effects, and whether the irrelevant spectral information can be accurately expressed as linear combinations of the original x variables. 

The GLS method was mentioned earher, as a preprocessing method that down-weights multivariate directions in the data that correspond to known interfering effects. However, it can also be used in a calibration transfer context, where directions in the data that correspond to instrumental differences are down-weighted. The use of GLS weighting for cahbration transfer is discussed in reference [116]. 

Raw signals from chemical sensors are rarely suitable for direct multivariate analysis. Some form of signal conditioning is always necessary before the input matrix is composed. Examples of preprocessing techniques used in the static and in the dynamic mode of multicomponent analysis are summarized in Table 10.1. They can be used as such or in combination. In higher-order sensors, where different transduction modes are used, the homogeneity of the input matrix is important. Thus, the matrix must contain data that are comparable in dimensions and that are commensurate. 

Preprocessing of instrument response data can be a critical step in the development of successful multivariate calibration models. Oftentimes, selection of an appropriate preprocessing technique can remove unwanted artifacts such as variable path lengths or different amounts of scatter from optical reflectance measurements. Preprocessing techniques can be applied to rows of the data matrix (by object) or columns (by variable). 

---