Supervised learning methods

Multiple linear regression is strictly a parametric supervised learning technique. A parametric technique is one which assumes that the variables conform to some distribution (often the Gaussian distribution) the properties of the distribution are assumed in the underlying statistical method. A non-parametric technique does not rely upon the assumption of any particular distribution. A supervised learning method is one which uses information about the dependent variable to derive the model. An unsupervised learning method does not. Thus cluster analysis, principal components analysis and factor analysis are all examples of unsupervised learning techniques. 

The Back-Propagation Algorithm (BPA) is a supervised learning method for training ANNs, and is one of the most common forms of training techniques. It uses a gradient-descent optimization method, also referred to as the delta rule when applied to feedforward networks. A feedforward network that has employed the delta rule for training, is called a Multi-Layer Perceptron (MLP). 

Genetic programming, a specific form of evolutionary computing, has recently been used for predicting oral bioavailability [23], The results show a slight improvement compared with the ORMUCS Yoshida-Topliss approach. This supervised learning method and other described methods demonstrate that at least qualitative (binned) predictions of oral bioavailability seem tractable directly from the structure. 

This section will focus on classification methods, or supervised learning methods, where a method is developed using a set of calibration samples and complete prior knowledge about the class membership of the samples. The development of any supervised learning method involves three steps ... 

In Section 8.5 it was shown that supervised learning methods can be used to build effective classification models, using calibration data in which the classes are known. However, if one has a set of data for which no class information is available, there are several methods that can be used to explore the data for natural clustering of samples. These methods can also be called unsupervised learning methods. A few of these methods are discussed below. 

Supervised learning methods - multivariate analysis of variance and discriminant analysis (MVDA) - k nearest neighbors (kNN) - linear learning machine (LLM) - BAYES classification - soft independent modeling of class analogy (SIMCA) - UNEQ classification Quantitative demarcation of a priori classes, relationships between class properties and variables... 

Supervised (learning) methods where a priori information is needed, for example, demarcation of a certain number of classes in the classification process. 

We therefore here describe the realisation of our development of diffuse reflectance/absorbance FT-IR spectroscopy as a quantitative tool for the rapid analysis of all samples of biotechnological and other interest, specifically by exploiting the ability of modem, supervised learning methods to take multivariate spectral inputs and map them directly to the concentration of one or more target determinands (see above and [70]), using as before [46] mainly mixtures of ampicillin and E. coli as a model system. 

One of the problems with regression analysis, and other supervised learning methods, is that they seek to fit a model. This may seem like a curious statement to make, to criticize a method for doing just what it is intended to do. The reason that this is a problem is that given sufficient opportunity to fit a model then regression analysis will find an equation to fit a data set. What is meant by sufficient opportunity It has been... 

The first two parts of this section describe supervised learning methods which may be used for the analysis of classified data. One technique, discriminant analysis, is related to regression while the other, SIMCA, has similarities with principal component analysis (PCA). The final part of this section discusses some of the conditions which data should meet when analysed by discriminant techniques. 

LDA models may also be used to identify important variables but here it should be remembered that discriminant functions are not unique solutions. Thus, the use of LDA for variable selection may be misleading. Whatever form of supervised learning method is used for the identification... 

This chapter has described some of the more commonly used supervised learning methods for the analysis of data discriminant analysis and its relatives for classified dependent data, variants of regression analysis for continuous dependent data. Supervised methods have the advantage that they produce predictions, but they have the disadvantage that they can suffer from chance effects. Careful selection of variables and test/training sets, the use of more than one technique where possible, and the application of common sense will all help to ensure that the results obtained from supervised learning are useful. 

Like most supervised learning methods, the goal of the decision tree methodology is to develop classification rules that determine the class of any object from the values of the object s attributes. In the case of decision trees, as the name implies, the classification rules are embodied in a knowledge representation fonnalism called a decision tree. This method has been used to derive structure-activity relationships and to learn classification rules for reactions. 

---