Linear learning machine

This is the simplest possible type of neuron, used here for didactic purposes and not because it is the configuration to be recommended. Let us suppose that for this isolated neuron w, = 1, Wj = 2 and 7=1. The line in Fig. 33.20 then gives the values of x, and Xj for which E = 7. All combinations of x, and Xj on and above the line will yield E > 7 and therefore lead to an output y, = 1 (i.e. the object is class K), all combinations below it toy, = 0. The procedure described here is equivalent to a method called the linear learning machine, which was one of the first supervised pattern recognition methods to be applied in chemometrics. It is further explained, including the training phase, in Chapter 44. 

The linear learning machine and the perceptron network 44.4.1 Principle... 

The perceptron-like linear networks were the first networks that were developed [3,4], They are described in an intuitive way in Chapter 33. In this section we explain their working principle as an introduction to that of the more advanced MLF networks. We explain the principle of these early networks by means of the Linear Learning Machine (LLM) since it is the best known example in chemistry. 

In the linear learning machine this rule is applied as follows ... 

In the class discrimination methods or hyperplane techniques, of which linear discriminant analysis and the linear learning machine are examples, the equation of a plane or hyperplane is calculated that separates one class from another. These methods work well if prior knowledge allows the analyst to assume that the test objects must... 

In 1969 Jurs, Kowalski and Isenhour published a first series of papers in ANALYTICAL CHEMISTRY, reporting the results of applying the linear learning machine to low resolution mass spectral data 21.22.2S) these studies was... 

Pattern recognition can be applied for the determination of structural features of unknown (monofunctional) compounds (Huber and Reich ). The information about the chemical structure is contained in a multidimensional gas-liquid retention data/stationary liquid phases set. The linear learning machine method is applied in a two step classification procedure. After the determination of a correction term, the skeleton number, a classification step for the determination of the functional group is executed. It is remarkable that 10 stationary phases are sufficient for the classification. 

Among the nonparametric techniques of pattern recognition, the linear learning machines have been only seldom used in food data analysis and it seems that this method is becoming obsolete. 

Bos et al. [94] compared the performance of ANNs for modelling the Cr-Ni-Fe system in quantitative XRF spectroscopy with the classical Rasberry-Heinrich model and a previously published method applying the linear learning machine in combination with singular value decomposition. They studied whether ANNs were able to model nonlinear relationships, and also their ability to handle non-ideal and noisy data. They used more than 100 steel samples with large variations in composition to calibrate the model. ANNs were found to be robust and to perform better than the other methods. 

Supervised learning methods - multivariate analysis of variance and discriminant analysis (MVDA) - k nearest neighbors (kNN) - linear learning machine (LLM) - BAYES classification - soft independent modeling of class analogy (SIMCA) - UNEQ classification Quantitative demarcation of a priori classes, relationships between class properties and variables... 

Canonical Correlation Analysis Principal Component Regressionb Classification and Regression Trees (CART) Linear Learning Machine Neural Networks Adaptive Least Squares Genetic Programming Logistic Regression... 

Linear learning machines can be expressed in a dual representation, enabling expression of the hypotheses as a linear combination of the training point (xj) so that the decision rule can be evaluated by using just inner products between the test points (x) and the training points ... 

If the membership of objects to particular clusters is known in advance, the methods of supervised pattern recognition can be used. In this section, the following methods are explained linear learning machine (LLM), discriminant analysis, A -NN, the soft independent modeling of class analogies (SIMCA) method, and Support Vector Machines (SVMs). 

Figure 5.25 Linear learning machine (LLM) representation of iodine data of Table 5.6 augmented by an additional dimension and separated by a straight-line boundary with the normal weight vector w.

The perceptron can be compared with the linear learning machine (Section 5.3). As demonstrated by Minsky and Papert (1969), certain problems cannot be solved by using a simple perceptron. As an example, there is the exclusive OR connection as given in Table 8.3 (cf. Figure 7.8). 

Discriminant analysis, also known as the linear learning machine, is intended for use with classified dependent data. The data may be measured on a nominal scale (yes/no, active/inactive, toxic/non-toxic) or an ordinal scale (1,2,3,4 active, medimn, inactive) or may be derived from continuous data by some rule (such as low if <10, high if > 10). The objective of... 

Linear discriminant analysis is equivalent to the linear learning machine. There are also procedures for non-linear discriminant analysis (as there are for non-linear regression) but these will not be considered here. 

As we have seen in chapter 1 and chapter 2, most of the data sets in chemistry and chemical engineering are nonlinear, while SVM is based on a linear learning machine with large margin. So it is necessary to have some nonlinear mapping techniques to make most of the data sets in chemistry or chemical engineering suitable to be treated by SVM. Kernel function is just an effective tool for this purpose. 

N. J. Nilsson, Linear Learning Machines. McGraw-Hill, New York, 1965. 

---