Cross-validation distance prediction

To develop a KNN model, a distance measure is selected and tlie optimal number of nearest neighbors is determined. It is recommended that K be selected using leave-one-out cross-validation applied to a training set. The outputs of the analysis arc the predicted classes for the training set and the goodness values. 

Another approach is to perform a cross-validation much like that for quantitative models. However, rather than predicting the constituent values as each sample is rotated out (and there are none to predict in discriminant analysis anyway), the Mahalanobis distance of each sample is predicted at every factor. The cross-validation procedure is basically the same remove a sample or set of samples, construct a Mahalanobis matrix for one factor, two factors, etc., and then predict the sample(s) left out against it. The samples are then returned to the training set, and a new set is removed. The process is continued until every sample has been rotated out once. 

Fio. 12. Average predicted Mahalanobis distance for a cross-validation of the spectra in Fig. 4. The plot reaches a value of 3.3 distances at 8 factors. However, the value at 9 factors is actually 3.0. An argument could be made for either number being correct. 

There is one additional method to use in determining outliers in discriminant analysis models to look at a plot of the predicted Mahalanobis distances (either from a cross-validation or self-prediction) to see if any samples stand out (Fig. 13). 

Fig. 13. Predicted Mahalanobis distances from a cross-validation of the training spectra in Fig. 4. The model was created with nine factors. Notice that sample No. 5 appears to be substantially different from the rest. However, upon further examination of the data, the spectrum of this sample had the largest baseline shift. Most likely the sample is fine and was left in...

It is evident that the approximation is expected to work well in situations (such as that near to equilibrium distances in Fig. 1.2) in which the electronic wave-functions are slowly changing functions of the nuclear coordinates, but to be much less valid when (as in the region of the crossing dashed lines of Fig. 1.2) the electronic wavefunctions change abruptly with nuclear motion. In the latter case, there may be some dynamic tendency for the states to preserve their electronic identity instead of following the changes predicted by the adiabatic BO approximation. 

---