Derivatives data preprocessing

Phase 2 - data preprocessing. There are many ways to process spectral data prior to multivariate image reconstruction and there is no ideal method that can be generally applied to all types of tissue. It is usual practice to correct the baseline to account for nonspecific matrix absorptions and scattering induced by the physical or bulk properties of the dehydrated tissue. One possible procedure is to fit a polynomial function to a preselected set of minima points and zero the baseline to these minima points. However, this type of fit can introduce artifacts because baseline variation can be so extreme that one set of baseline points may not account for all types of baseline variation. A more acceptable way to correct spectral baselines is to use the derivatives of the spectra. This can only be achieved if the S/N of the individual spectra is high and if an appropriate smoothing factor is introduced to reduce noise in the derivatized spectra. Derivatives serve two purposes they minimize broad... 

On the Data Preprocessing page (Fig. 11.23) select one of the two data preprocessing methods either the derivative or the vector normalization. The vector normalization causes the Euclidean distance to fall within an interval of 0 and 2. 

Second derivative ratio preprocessing has also been demonstrated in the Norris regression method. Empirically, this data preprocessing technique has proved useful for some applications. The following relationship designates a second derivative ratio with two different center wavelengths ... 

Niculescu et al. have reported the application of a related probabilistic neural net to bioactive prediction (136). These authors investigated the connection between the data preprocessing strategy and kernel choiee on the quality of the derived models. Ajay et al. also employed Bayesian methods to design a CNS-active library (97). A neural network trained using Bayesian methods was trained on CNS-aetive and CNS-inaetive data and correctly predicted up to 92% and 71% accuracy on the aetives and inactives. They used the method to generate a small library of potentially CNS-active moleeules amenable to combinatorial synthesis. 

If the x-data of an object are time-series or digitized data from a continuous spectrum (infrared, IR near infrared, NIR) then smoothing and/or transformation to first or second derivative may be appropriate preprocessing techniques. Smoothing tries to reduce random noise and thus removes narrow spikes in a spectrum. Differentiation extracts relevant information (but increases noise). In the first derivative an additive baseline is removed and therefore spectra that are shifted in parallel to other... 

The movement of a fiber-optic probe and/or the hbers leading to/from a probe or cell can also induce baseline shifts in the recorded spectra. These shifts are usually minor and can be eliminated with proper preprocessing of the data (e.g. derivatives). (A detailed description of preprocessing techniques can be found in Chapter 12)... 

This filtering preprocessing method can be used whenever the variables are expressed as a continuous physical property. One example is dispersive or Fourier-Transform spectral data, where the spectral variables refer to a continuous series of wavelength or wavenumber values. In these cases, derivatives can serve a dual purpose (I) they can remove baseline offset variations between samples, and (2) they can improve the resolution of overlapped spectral features. 

Selected Variable Plot (Model and Variable Diagnostic) Figure 5.79 shows the three selected variables for the first derivative preprocessed data. 

It is also possible to derive a vector of regression coefficients that applies directly to the (preprocessed) measured data. This vector can be calculated as... 

After preprocessing the data, the search begins to find the features in the data that show the largest differences between patterns. This process is very much dependent on the type of detectors in the detection system but may involve comparing the relative amplitudes of the different detectors in the array, the derivative of the response, or even mathematical transforms of the data to select which features show the most differentiation between the patterns of different analytes. For our examples described in Section 5.3., we chose to use 120-data point analyte signature patterns sampled from the entire shape of the signal from a detector array of four cantilevers with coatings described in Section 3.5. 

The previously discussed standardization methods require that calibration-transfer standards be measured on both instruments. There may be situations where transfer standards are not available, or where it is impractical to measure them on both instruments. In such cases, if the difference between the two instruments can be approximated by simple baseline offsets and path-length differences, preprocessing techniques such as baseline correction, first derivatives, or MSC can be used to remove one or more of these effects. In this approach, the desired preprocessing technique is applied to the calibration data from the primary instrument before the calibration model is developed. Prediction of samples from the primary or secondary instrument is accomplished simply by applying the identical preprocessing technique prior to prediction. See Section 5.9 for a brief overview of preprocessing methods and Chapter 4 for a more detailed discussion. A few methods are briefly discussed next. 

Expanding the experimental data to a larger sample, we selected Type 2 days (for which data were available during the 1968 smog season). The statistical preprocessing program derived half-hourly means... 

The extraction of the eigenvectors from a symmetric data matrix forms the basis and starting point of many multivariate chemometric procedures. The way in which the data are preprocessed and scaled, and how the resulting vectors are treated, has produced a wide range of related and similar techniques. By far the most common is principal components analysis. As we have seen, PCA provides n eigenvectors derived from a. nx n dispersion matrix of variances and covariances, or correlations. If the data are standardized prior to eigenvector analysis, then the variance-covariance matrix becomes the correlation matrix [see Equation (25) in Chapter 1, with Ji = 52]. Another technique, strongly related to PCA, is factor analysis. ... 

---