Parametric classification methods

Theory. SIMCA is a parametric classification method introduced by Wold (29), which supposes that the objects of a given class are normally distributed. The particularity of this PCA-based method is that one model is built for each class separately, that is, disjoint class modeling is performed. The algorithm starts by determining the optimal number of PCs for each individual model with CV. The resulting PCs are then used to define a hypervolume for each class. The boundary around one group of objects is then the confidence limit for the residuals of all objects determined by a statistical T-test (30, 31). The direction of the PCs and the limits established for these PCs define the model of a class (Fig. 13.13). 

PCA is not only used as a method on its own but also as part of other mathematical techniques such as SIMCA classification (see section on parametric classification methods), principal component regression analysis (PCRA) and partial least-squares modelling with latent variables (PLS). Instead of original descriptor variables (x-variables), PCs extracted from a matrix of x-variables (descriptor matrix X) are used in PCRA and PLS as independent variables in a regression model. These PCs are called latent variables in this context. 

The probability density functions cannot be stored point by point because they depend on many (d) variables. Therefore several parametric classification methods assume Gaussian distributions and the estimated parameters of these distributions are used to specify a decision function. Another assumption of parametric classifiers are statistically independent pattern features. 

A statistical meaningful estimation of probability densities requires very large data sets. Therefore, chemical applications of parametric classification methods always include assumptions which are often not fulfilled or cannot be proved. A severe assumption is the statistical independence of the pattern components which is certainly often not satisfied. Generation of new independent features is usually too laborious (Chapter 10). 

The Bayesian approach is one of the probabilistic central parametric classification methods it is based on the consistent apphcation of the classic Bayes equation (also known as the naive Bayes classifier ) for conditional probabihty [34] to constmct a decision rule a modified algorithm is explained in references [105, 109, 121]. In this approach, a chemical compound C, which can be specified by a set of probability features (Cj,...,c ) whose random values are distributed through all classes of objects, is the object of recognition. The features are interpreted as independent random variables of an /w-dimensional random variable. The classification metric is an a posteriori probability that the object in question belongs to class k. Compound C is assigned to the class where the probability of membership is the highest. 

The classification methods discussed in the previous section are all based on statistical tests wliich require normal data distribution. If this condition is not fulfilled the so-called non-probabihstic , non-parametric or heuristic classification techniques must be used. These techniques are also frequently referred to as pattern recognition methods. They are based on geometrical and not on statistical considerations, starting from a representation of the compounds... 

As an alternative to the classification methods hsted above, logistic regression can be used as a parametric approach. A logistic regression model cannot be used if the number of predictors exceeds the number of observations. In this case, a variable selection needs to be conducted in order to utilize logistic regression. 

The accuracy of the average risk of a classification can be computed in principle if the statistics of the data are known. However, the uncertainties mentioned considerably limit the computation of risk in practical applications. Therefore, the usefulness of a parametric method for a practical classification problem can be examined - as for other classification methods - only empirically. 

This classification method was first applied to chemical problems by Franzen and Hillig C86, 87, 108D. Although many simplifications have been introduced into this maximum likelihood method a considerable computational effort is necessary for the training and application of such parametric classifiers. However, the effort is much smaller if binary encoded patterns are used (Chapter 5.4). 

The aim of supervised classification is to create rules based on a set of training samples belonging to a priori known classes. Then the resulting rules are used to classify new samples in none, one, or several of the classes. Supervised pattern recognition methods can be classified as parametric or nonparametric and linear or nonlinear. The term parametric means that the method makes an assumption about the distribution of the data, for instance, a Gaussian distribution. Frequently used parametric methods are EDA, QDA, PLSDA, and SIMCA. On the contrary, kNN and CART make no assumption about the distribution of the data, so these procedures are considered as nonparametric. Another distinction between the classification techniques concerns the... 

Theory. LDA, a popular method for supervised classification, was introduced by Fisher in 1936 (21). The goal of this method is to classify the samples, establishing a linear function based on the variables X (i ranges from 1 to n, the number of considered variables), which separates the classes existing in the training set (Fig. 13.8). Classification is based on the interclass discrimination (22). It is a parametric method because the method assumes that the distribution of the samples in the classes is Gaussian. 

QDA is identical to LDA, but this method is based on a quadratic classification curve instead of a straight line. The data must be normally distributed as for the LDA method. QDA is thus a linear parametric method. 

Quite a variety of different methods can and have been apphed in QSAR work for the evaluation of classification rules [74]. These methods may roughly be divided into two categories, namely parametric or statistical and non-parametric or heuristic techniques. While class separation in the parametric techniques is... 

Regression analysis (black box models) i.e., statistical methods that permit the approximation of functions and the classification of data using non-parametric methods (application specific). 

Pattern recognition algorithms are often categorized as paramet ri c or non-pa ramet r i c. Parametric methods require the knowledge of the "statistics of the classification problem". If the probability of each class is known at any location in the d-dimensional pattern space then an optimum classification of an unknown pattern can be made by selection of the "most probable" class at that point. The statistics of the classification problem is estimated by the use of a training set which should be as large as possible. In practical problems the actual statistics can never be known exactly because this would require that all possible measurements had been performed. The available data are never fully representative of a problem and therefore only less than optimum classifications can be achieved. 

The literature of multivariate classification shows that several types of methods have found utility in application to chemical problems. Excellent discussions of the major methods can be found in Strouf ° and Tou and Gon-zalez. The most frequently used methods include parametric approaches involving linear and quadratic discriminant analysis based on the Bayesian approach,nonparametric linear discriminant development methods,and those methods based on principal components analysis such as SIMCA (Soft Independent Modeling by Class Analogy). 

The distance method is a geometric central parametric method its modified algorithm is described in references [105, 109]. In this case, the object of classification (chemical compound C) is defined by a set of determined features (Cj,..., c whose values are interpreted as coordinates of a point in a multidimensional space of m dimension. The classification metric is the distance from the object in question to the geometric center of class k. Compound C belongs to the class placed at a shorter distance. 

The local distribution method is one combination method using the geometric local nonparametric method in parallel to a probabilistic central parametric method for decision rale constraction. The algorithm was first deseiibed in [109] and later modified as described in [105]. Two metrics serve as classification metrics the similarity coefficient of the features of the object to be predicted and class k objects in /M-dimensional space, and the probabihty that the object of interest belongs to the subclass of similar objects in class k. Componnd C is assigned to the class with the greatest local probabihty that the componnd belongs to the structurally similar subclass. 

The central parametric methods of classification are based on generalized information about all of the objects in the training set, so when applying these methods, one generally takes the most significant regularities that are typical of this type of activity into consideration. Conversely, local nonparametric methods consider the characteristics of objects that are closest to the predicted stmcture, so they mostly... 

These fall into two categories parametric and nonparametric. The former assumes that the samples for the K classes derive from some known distribution (usually multivariate normal), whereas the latter is distribution-free. There is no best method the choice of the classifier depends both on the problem and on the property that we want to optimize. Most frequently, we minimize the classification error, in particular By for the validation set, but this need not be the only choice. Our current concern is with the classification of spectra. In the following, we shall assume that the original, L-dimensional feature space was already reduced to an M-dimensional space, M L (M-space). 

Pattern recognition can be categorized into parametric and non-parametric methods. Non-parametric methods do not make any assumptions about the underlying statistics of the data. Parametric methods estimate probability densities for class memberships (or a response variable) and then apply a classification rule to classify unknown objects or to predict the response. 

---