Variance feature selection

Feature Selection. Once a data set has been established to be linearly separable, then variance feature selection (50) can be used to discard the least useful descriptors and thereby focus on the more useful descriptors. Several routines are implemented... 

There are many advantages in using this approach to feature selection. First, chance classification is not a serious problem because the bulk of the variance or information content of the feature subset selected is about the classification problem of interest. Second, features that contain discriminatory information about a particular classification problem are usually correlated, which is why feature selection methods using principal component analysis or other variance-based methods are generally preferred. Third, the principal component plot... 

When feature selection is used to simplify, because of the large number of variables, methods must be simple. The univariate criterion of interclass variance/intraclass variance ratio (in the different variants called Fisher weights variance weights or Coomans weights is simple, but can lead to the elimination of variables with some discriminant power, either separately or, more important, in connection with other variables (Fig. 36). 

This routine is the one ordinarily used to train a set of slightly different weight vectors to be used for the variance method of nonparametric feature selection. 

One group of feature selection methods uses the statistics of the data (means, variances) to select the most important features. The features are ranked according to their importance and less important features are discarded. 

If alL features are binary encoded (x = 0 or 1) some simplifications and specialities exist. One possible feature selection method determines those features which have maximum variance among the a posteriori probabilities as calculated by the Bayes rule C170, 171, 3533. 

The aim of training is to obtain sets of spectral data that can be used to determine decision rules for the classification of each pixel in the whole image data set. The training data for each class must be representative of all data for that class. Each training site consists of many pixels, conventionally it is taken that if there are n number of bands the number of pixels in each band is n+1. The mean, standard deviation, variance, minimum value, maximum value, variance-covariance matrix and correlation matrix for training classes are calculated, which represent the fundamental information on the spectral characteristics of all classes. Since for selection of appropriate bands only this information is not enough, thus feature selection is used. The training sites are presented on true color map bands 1, 2, and 3 for 13 classes (Fig. 16). 

In unsupervised cases (only an X-matrix is available) a simple criterion for selecting features is the variance. Features with a low variance are considered to possess less information and are eliminated. 

The selection of variables could separate relevant information from unwanted variability and at the same time allows data compression, that is more parsimonious models, simplification or improvement of model interpretation, and so on. Although many approaches can be used for features selection, in this work, a wavelet-based supervised feature selection/classification algorithm, WPTER [12], was applied. The best performing model was obtained using a daubechies 10 wavelet, a maximum decomposition level equal to 10, between-class/within-class variance ratio criterion for the thresholding operation and the percentage of selected coefficients equal to 2%. Six wavelet coefficients were selected, belonging to the 4th, 5th, 6th, 8th, and 9th levels of decomposition. 

One way of selecting discriminating features is to compare the means and the variances of the different variables. Variables with widely different means for the classes and small intraclass variance should be of value and, for a binary discrimination, one therefore selects those variables for which the expression... 

Note The used variable sets are 14 modulo-14 features (autoscaled) 2 and 3 PCA scores calculated from the autoscaled modulo-14 features peak intensities at 14 selected mass numbers (with maximum variances of the peak intensities) 50 mass spectral features. The numbers of correct predictions are from a leave-one-out test n is the number of spectra in the five DBE groups... 

---