Supervised learning techniques

Multiple linear regression is strictly a parametric supervised learning technique. A parametric technique is one which assumes that the variables conform to some distribution (often the Gaussian distribution) the properties of the distribution are assumed in the underlying statistical method. A non-parametric technique does not rely upon the assumption of any particular distribution. A supervised learning method is one which uses information about the dependent variable to derive the model. An unsupervised learning method does not. Thus cluster analysis, principal components analysis and factor analysis are all examples of unsupervised learning techniques. 

Discriminant emalysis is a supervised learning technique which uses classified dependent data. Here, the dependent data (y values) are not on a continuous scale but are divided into distinct classes. There are often just two classes (e.g. active/inactive soluble/not soluble yes/no), but more than two is also possible (e.g. high/medium/low 1/2/3/4). The simplest situation involves two variables and two classes, and the aim is to find a straight line that best separates the data into its classes (Figure 12.37). With more than two variables, the line becomes a hyperplane in the multidimensional variable space. Discriminant analysis is characterised by a discriminant function, which in the particular case of hnear discriminant analysis (the most popular variant) is written as a linear combination of the independent variables ... 

Supervised learning techniques are applied when one is seeking to classify a set of data with the aid of previously defined patterns. In analysis one of the most investigated problems of this sort has been the interpretation of mass spectra, both with respect to identifying components of a mixture and for naming an unknown compound on the basis of fragmentation patterns. 

As a final remark it should be realized that when using some Supervised Learning techniques like SIMCA, the scaling of the data set is carried out only over the samples belonging to the same class (separate scaling). This is due because the own fundamentals of the methodology and has a beneficial effect on the classification (Derde et al., 1982). 

Weighting features in Supervised Learning techniques can be then extracted from its relative importance in discriminating classes pairwise. The largest weight corresponds to the most important feature. The most common weighting factors are ... 

The decision tree classifier is chosen for its favorable tradeoff between performance and implementation simplicity. Classification using DT is a supervised learning technique, the input of the learning algorithm is a set of known data and the output is a tree model similar to the ones shown in Figure 5. Once the tree is defined, the classification of new inputs starts at the root decision node of the tree and terminates at one of the leaf nodes that represent a specific class, passing by intermediate decision nodes. 

The SIMCA method is based on the construction of principal components (PCs) which effectively fit a box (or hyper-box in N dimensions) around each class of samples in a data set. This is an interesting application of PCA, an unsupervised learning method, to different classes of data resulting in a supervised learning technique. The relationship between SIMCA and PLS, another supervized principal components method, can be seen clearly by reference to Section 7.3. 

---