Outlier spectra

Identify outlier spectra within the rows or columns that do not conform to a parent pattern . 

These error values may be depicted graphically with the number n of the corresponding training spectrum on the abscissa (cf. Fig. 22.10). This graph permits an easy detection of outlier spectra. 

Outlier spectrum, n - a spectrum whose analysis by a multivariate model represents an extrapolation of the model. 

Figure 36.5 shows the scores plots for PC2 v. PCI (A) and PC4 v. PC3 (B). Such plots are useful in indicating a possible clustering of samples in subsets or the presence of influential observations. Again, a spectrum with a spike may show up as an outlier for that sample in one of the scores plots. If outliers are indicated, one should try and identify the cause of the outlying behaviour. Only when a satisfactory explanation is found can the outlier be safely omitted. In practice, one will... 

Outlier detection limits, n - the limiting value for application of an outlier detection method to a spectrum, beyond which the spectrum represents an extrapolation of the calibration model. 

Raw Measurement Plot In multivariate calibration, it is normally not necessary to plot the prediction data if the outlier detection technique has not flagged the sample as an outlier. However, with MLR, the outlier detection methods are not as robust as with the full-spectrum techniques (e.g., CLS, PLS, PCR) because few variables are considered. Figure 5.75 shows all of the prediction data with the variables used in the modeling noted by vertical lines. One sample appears to be unusual, with an extra peak centered at variable 140. The prediction of this sample might be acceptable because the peak is not located on the variables used for the models. However, it is still suspect because the new peak is not expected and can be an indication of other problems. 

Although gross outliers should be deleted in advance (see the initial paragraphs in Section 4.4), it is likely to occur that some sample with some particular characteristic (spectrum, noise, etc.) shows a suspicious or different behaviour. Two plots inspecting the space of the spectra and the relationship of the X-space with the Y-space will be useful here. The X-spectral space is searched for by the z-z scores plot". Here the scores of the second, third, fourth, etc.. 

X-variables. This leads to the presence of model residuals (E in Equations 8.19 and 8.35). The residuals of the model can be used to indicate the nature of unmodeled information in the calibration data. For process analytical spectroscopy, plots of individual sample residuals versus wavelength ( residual spectra ) can be used to provide some insight regarding chemical or physical effects that are not accounted for in the model. In cases where a sample or variable outlier is suspected in the calibration data, inspection of that sample or variable s residual can be used to help determine whether the sample or variable should be removed from the calibration data. When a model is operating on-line, the X-residuals of prediction (see Equation 8.55) can be used to determine whether the sample being analyzed is appropriate for application to a quantitative model (see Section 8.4.3). In addition, however, one could also view the prediction residual vector ep as a profile (or residual spectrum ) in order to provide some insight into the nature of the prediction sample s inappropriateness. 

This approach to calibration, although widely used throughout most branches of science, is nevertheless not always appropriate in all applications. We may want to answer the question can the absorbance in a spectrum be employed to determine the concentration of a compound . It is not the best approach to use an equation that predicts the absorbance from the concentration when our experimental aim is the reverse. In other areas of science the functional aim might be, for example, to predict an enzymic activity from its concentration. In the latter case univariate calibration as outlined in this section results in the correct functional model. Nevertheless, most chemists employ classical calibration and provided that the experimental errors are roughly normal and there are no significant outliers, all the different univariate methods should result in approximately similar conclusions. 

Axxrid the line frequency and harmonics Modem impedance instniments provide very effective filters for stochastic noise, but these filters are generally inadequate for measiuements conducted at the line frequency. The resulting meastuements generally appear as outliers in an impedance spectrum, and such outliers have a profovmd impact on nonlinear regression used to extract parameters from the data. Measurement of impedance should be avoided at line frequency and its first harmonic, i.e., 60 5 Hz and 120 5 Hz in the United States and 50 5 Hz and 100 5 Hz in Europe. 

Avoid the line frequency and harmonics As discussed above, measurements made at the line frequency or its first harmonic typically have a significant error that will appear as an outlier when compared to the rest of the spectrum. Measurement within 5 Hz of the line quency and its first harmonic should be avoided. 

Seventeen mixtures were generated from the same two components a and b used above and the relative content of a and b in mixtures were designed using coordinates of points on a circle (Table 2) this simulated data set was denoted as CYCOO. Then the spectrum of mixture No. 2 was contaminated by several Gaussian bands and used as an outlier. The contaminated data set was called CYCO1. 

First order (vector, Outlier detection, spectrum) interference... 

The largest of these simply describes the average spectrum of the seeds. The second and third components can be used in scatter plots to show detailed information of outliers and clustering in the mother and father spaces. Figure 10.63 shows the scatter plot of b3 and b2 for the fathers. For the mothers, Figure 10.64 shows the scatter plot of a3 and a2. This shows that genetic information is present in the spectra. According to the MANOVA results, the mothers information should be more reliable than that of the fathers. 

The presence of outliers changes the statistical properties of the data, such as the autocorrelation function and power spectrum. In the filtering techniques described so far, it is assumed that only stationary Gaussian... 

The data couples (Qxp > Cpred) in Fig- 22.11 may be evaluated by all statistical procedures. Cross validation is a very powerful method for outlier detection and optimization of calibration models. If the predicted concentrations obey the necessary quality standard for all but one training spectrum, the latter can be regarded as an outlier. 

---