Neural networks Perceptron

Artificial Neural Networks (ANNs) attempt to emulate their biological counterparts. McCulloch and Pitts (1943) proposed a simple model of a neuron, and Hebb (1949) described a technique which became known as Hebbian learning. Rosenblatt (1961), devised a single layer of neurons, called a Perceptron, that was used for optical pattern recognition. 

Chapter 10 covers another important field with a great overlap with CA neural networks. Beginning with a short historical survey of what is really an independent field, chapter 10 discusses the Hopfield model, stochastic nets, Boltzman machines, and multi-layered perceptrons. 

While, as mentioned at the close of the last section, it took more than 15 years following Minsky and Papert s criticism of simple perceptrons for a bona-fide multilayered variant to finally emerge (see Multi-layeved Perceptrons below), the man most responsible for bringing respectability back to neural net research was the physicist John J, Hopfield, with the publication of his landmark 1982 paper entitled Neural networks and physical systems with emergent collective computational abilities [hopf82]. To set the stage for our discussion of Hopfield nets, we first pause to introduce the notion of associative memory. 

The basic component of the neural network is the neuron, a simple mathematical processing unit that takes one or more inputs and produces an output. For each neuron, every input has an associated weight that defines its relative importance, and the neuron simply computes the weighted sum of all the outputs and calculates an output. This is then modified by means of a transformation function (sometimes called a transfer or activation function) before being forwarded to another neuron. This simple processing unit is known as a perceptron, a feed-forward system in which the transfer of data is in the forward direction, from inputs to outputs, only. 

A neural network consists of many neurons organized into a structure called the network architecture. Although there are many possible network architectures, one of the most popular and successful is the multilayer perceptron (MLP) network. This consists of identical neurons all interconnected and organized in layers, with those in one layer connected to those in the next layer so that the outputs in one layer become the inputs in the subsequent... 

Neural networks have been introduced in QSAR for non-linear Hansch analyses. The Perceptron, which is generally considered as a forerunner of neural networks has been developed by the Russian school of Rastrigin and coworkers [62] within the context of QSAR. The learning machine is another prototype of neural network which has been introduced in QSAR by Jurs et al. [63] for the discrimination between different types of compounds on the basis of their properties. 

All of the studies above have used back propagation multilayer perceptrons and many other varieties of neural network exist that have been applied to PyMS data. These include minimal neural networks,117119 radial basis functions,114120 self-organizing feature maps,110121 and autoassociative neural networks.122123... 

It is easy to construct a network of perceptrons by bolting them together so that the outputs of some of them form the inputs of others, but in truth it is hardly worth the effort. The perceptron is not just simple, it is too simple. A network of perceptrons constructed manually can perform a few useful tasks, but it cannot learn anything worthwhile, and since learning is the key to a successful neural network, some modification is needed. 

Schierle and Otto [63] used a two-layer perceptron with error back-propagation for quantitative analysis in ICP-AES. Also, Schierle et al. [64] used a simple neural network [the bidirectional associative memory (BAM)] for qualitative and semiquantitative analysis in ICP-AES. 

Catasus et al. [67] studied two types of neural networks traditional multilayer perceptron neural networks and generalised regression neural networks (GRNNs) to correct for nonlinear matrix effects and long-term signal drift in ICP-AES. 

Fig. 10.8 (a) Example of common neural net (perceptron) architecture. Here one hidden layer Neural Networks (NNs) is shown (Hierlemann et al., 1996). (b) A more sophisticated recurrent neural network utilizing adjustable feedback through recurrent variables, (c) Time-delayed neural network in which time has been utilized as an experimental variable... 

Three commonly used ANN methods for classification are the perceptron network, the probabilistic neural network, and the learning vector quantization (LVQ) networks. Details on these methods can be found in several references.57,58 Only an overview of them will be presented here. In all cases, one can use all available X-variables, a selected subset of X-variables, or a set of compressed variables (e.g. PCs from PCA) as inputs to the network. Like quantitative neural networks, the network parameters are estimated by applying a learning rule to a series of samples of known class, the details of which will not be discussed here. 

Figure6.25 Schematicdrawingofan artificial neural network with a multilayer perceptron topology, showing the pathways from the input Xj to the output y , and the visible and hidden node layers.

Among the most wide-spread neural networks are feedforward networks, namely multilayer perceptron (MLP). This network type has been proven to be universal function approximators [11], Another important feature of MLP is the ability to generalization. Therefore MLP can be powerful tool for design of intrusion detection systems. 

We describe the intrusion detection system, which consists of two different neural networks. The first neural network is nonlinear PCA (principal component analysis) network, which permits to identify normal or anomalous system behavior. The second one is multilayer perceptron (MLP), which can recognize type of attack. 

The rest of the paper is organized as follows. The Section 2 describes attack classification and training data set. In the Section 3 the intrusion detection system is described, based on neural network approach. Section 4 presents the nonlinear PCA neural network and multilayer perceptron for identification and classification of computer network attack. In Section 5 the results of experiments are presented. Conclusion is given in Section 6. 

The neural network for identification is nonlinear PCA (NPCA) network [18]. As input data in this case, four features service, duration, src bytes, and dst bytes are used. The neural network for recognition is multilayer perceptron. In this case, all of the listed features above (Table 3) are used as input data. Such a system permits to identify and recognize the network attacks. 

Let s consider the neural network for recognition of attack. This network is multilayer perceptron with 6 input units, 40 hidden units and 23 output... 

Fig. 17. Use of a multilayer perceptron-type artificial neural network to analyze an interferometric image of...

Figure 5.3 Simple multilayered perceptron of an artificial neural network...

The characters are first normalized by rotating the original scanned image to correct for scanning error and by combinations of scaling under sampling and contrast and density adjustments of the scanned characters. In operation, the normalized characters are then presented to a multilayer perceptron neural network for recognition the network was trained on exemplars of characters form numerous serif and sans serif fonts to achieve font invariance. Where the output from the neural network indicates more than one option, for example 5 and s, the correct interpretation is determined from context. 

Usually complex structures with more than 15 layers are employed, called the multilayer perceptron (MLP). Some of the commercial programs which have been used to fit tableting parameters are INForm (Intelligensys, Billingham Teesside), CAD/Chem (AI Ware, Cleveland, OH), which is no longer commercially available, and the Neural Network Toolbox of MATLAB (MathWorks, Natick, MA). 

In the previous chapter a simple two-layer artificial neural network was illustrated. Such two-layer, feed-forward networks have an interesting history and are commonly called perceptrons. Similar networks with more than two layers are called multilayer perceptrons, often abbreviated as MLPs. In this chapter the development of perceptrons is sketched with a discussion of particular applications and limitations. Multilayer perceptron concepts are developed applications, limitations and extensions to other kinds of networks are discussed. 

The field of artificial neural networks is a new and rapidly growing field and, as such, is susceptible to problems with naming conventions. In this book, a perceptron is defined as a two-layer network of simple artificial neurons of the type described in Chapter 2. The term perceptron is sometimes used in the literature to refer to the artificial neurons themselves. Perceptrons have been around for decades (McCulloch Pitts, 1943) and were the basis of much theoretical and practical work, especially in the 1960s. Rosenblatt coined the term perceptron (Rosenblatt, 1958). Unfortunately little work was done with perceptrons for quite some time after it was realized that they could be used for only a restricted range of linearly separable problems (Minsky Papert, 1969). 

---