Classification rules

Many of the tools are aimed at classification and prediction problems, such as the handwriting example, where a training set of data vectors for which the property is known is used to develop a classification rule. Then the rule can be appHed to a test set of data vectors for which the property is... 

Supervised Learning. Supervised learning refers to a collection of techniques ia which a priori knowledge about the category membership of a set of samples is used to develop a classification rule. The purpose of the rule is usually to predict the category membership for new samples. Sometimes the objective is simply to test the classification hypothesis by evaluating the performance of the rule on the data set. 

EC Reclassifies breast implants as Class 111 devices by way of derogation from the general classification rules... 

Derivation of a classification rule, using the training set. This is the subject of Section 33.2. 

Validation of the classification rule, using an independent test set. This is described in more detail in Section 33.4. 

There are many types of pattern recognition which essentially differ in the way they define classification rules. In this section, we will describe some of the approaches, which we will then develop further in the following sections. We will not try to develop a classification of pattern recognition methods but merely indicate some characteristics of the methods, that are found most often in the chemometric literature and some differences between those methods. 

When working with standardized data Wg = 0. The coefficients w and W2 are derived in a way described later, such that D = 0 in point O and D > 0 for objects belonging to L and > < 0 for objects of K. This then is the classification rule. 

Friedman [12] introduced a Bayesian approach the Bayes equation is given in Chapter 16. In the present context, a Bayesian approach can be described as finding a classification rule that minimizes the risk of misclassification, given the prior probabilities of belonging to a given class. These prior probabilities are estimated from the fraction of each class in the pooled sample ... 

The basis of classification is supervised learning where a set of known objects that belong unambiguously to certain classes are analyzed. From their features (analytical data) classification rules are obtained by means of relevant properties of the data like dispersion and correlation. 

There are various different ways for finding classification rules. The main approaches are based on... 

Maximizing the posterior probabilities in case of multivariate normal densities will result in quadratic or linear discriminant rules. However, the mles are linear if we use the additional assumption that the covariance matrices of all groups are equal, i.e., X = = Xk=X- In this case, the classification rule is based on linear discriminant scores dj for groups j... 

FIGURE 5.4 Linear discriminant scores dj for group j by the Bayesian classification rule based on (Equation 5.2). mj, mean vector of all objects in group j Sp1, inverse of the pooled covariance matrix (Equation 5.3) x, object vector (to be classified) defined by m variables Pj, prior probability of group j. 

Here, y is a tuning parameter taking values in the interval [0, 1] that adjusts the importance of the score and orthogonal distance for the classification. One can use CV to find the optimum value of y. Equation 5.24 results in a score value of an object jc for each group. A soft classification rule defines that an object jc is assigned to... 

One has to be careful with the use of the misclassification error as a performance measure. For example, assume a classification problem with two groups with prior probabilities pi = 0.9 and p2 = 0.1, where the available data also reflect the prior probabilities, i.e., nx k, npi and n2 np2. A stupid classification rule that assigns all the objects to the first (more frequent) group would have a misclassification error of about only 10%. Thus it can be more advisable to additionally report the misclassification rates per group, which in this case are 0% for the first group but 100% for the second group which clearly indicates that such a classifier is useless. 

Test error The classification rule is derived from the whole calibration set with a certain parameter choice. Then the mle is applied to the test set, and the test error is the resulting misclassification error of the test set. Note that in principle it would be sufficient to compute the test error only for the optimal parameter choice. 

Developing a classification rule This step requires the known class membership values for all calibration samples. Classification rules vary widely, but they essentially contain two components ... 

The KNN method [77] is probably the simplest classification method to understand. Once the model space and distance measure are defined, its classification rule involves rather simple logic ... 

Unlike other classification methods, the PLS-DA method explicitly determines relevant multivariate directions in the data (the PLS latent variables) that optimize the separation of known classes. Second, unlike KNN, the classification rule for PLS-DA is based on statistical analysis of the prediction values, which allows one to apply prior knowledge regarding the expected analytical response distributions of the different classes. Furthermore, PLS-DA can handle cases where an unknown sample belongs to more than one class, or to no class at all. 

HCA is a common tool that is used to determine the natural grouping of objects, based on their multivariate responses [75]. In PAT, this method can be used to determine natural groupings of samples or variables in a data set. Like the classification methods discussed above, HCA requires the specification of a space and a distance measure. However, unlike those methods, HCA does not involve the development of a classification rule, but rather a linkage rule, as discussed below. For a given problem, the selection of the space (e.g., original x variable space, PC score space) and distance measure (e.g.. Euclidean, Mahalanobis) depends on the specific information that the user wants to extract. For example, for a spectral data set, one can choose PC score space with Mahalanobis distance measure to better reflect separation that originates from both strong and weak spectral effects. 

Establish training sets. Derive classification rules. Select features. 

Once the training sets have been established, it is necessary to obtain data on them relevant to classification of subsequent samples. These data are the basis of the classification rules to be derived. These samples of unknown class assignment are known as the test samples or collectively as the test set. The training set(s) and test set are tabulated with their data, as in Figure 1. 

---