Artificial neurons output function

Figure 9-13. Artificial neuron the signals x, are weighted (with weights IV,) and summed to produce a net signal Net. This net signal is then modified by a transfer function and sent as an output to other neurons,...

A biological neuron can be active (excited) or inactive (not excited). Similarly, the artificial neurons can also have different activation status. Some neurons can be programmed to have only two states (active/inactive) as the biological ones, but others can take any value within a certain range. The final output or response of a neuron (let us call it a) is determined by its transfer function, f, which operates on the net signal (Netj) received by the neuron. Hence the overall output of a neuron can be summarised as ... 

Our choice for the non-linear system approach to PARC is the ANN. The ANN is composed of many neurons configured in layers such that data pass from an input layer through any number of middle layers and finally exit the system through a final layer called the output layer. In Fig. 4 is shown a diagram of a simple three-layer ANN. The input layer is composed of numeric scalar data values, whereas the middle and output layers are composed of artificial neurons. These artificial neurons are essentially weighted transfer functions that convert their inputs into a single desired output. The individual layer components are referred to as nodes. Every input node is connected to every middle node, and every middle node is connected to every output node. 

Neural network has been widely used in fields of function approximation, pattern recognition, image dealing, artificial intelligence, optimization and so on [26, 102]. Multilayer feed forward artificial neural network is a major type of the neural network which is connected by input layer, one or more output layers and hidden layers in a forward way. Each layer is composed of many artificial neurons. The output of previous layer neurons is the input of the next layer as shown in Fig. 2.6. 

Fig. 1 Scheme of an artificial neuron. First, several input numbers x, are added. Then a function/ is applied to this sum to yield the output y. 

The ANNs were developed in an attempt to imitate, mathematically, the characteristics of the biological neurons. They are composed by intercoimected artificial neurons responsible for the processing of input-output relationships, these relationships are learned by training the ANN with a set of irqmt-output patterns. The ANNs can be used for different proposes approximation of functions and classification are examples of such applications. The most common types of ANNs used for classification are the feedforward neural networks (FNNs) and the radial basis function (RBF) networks. Probabilistic neural networks (PNNs) are a kind of RBFs that uses a Bayesian decision strategy (Dehghani et al., 2006). 

Figure 9.6. Artificial neuron or node, p, a, and w represent the input, output, and weight, respectively. n is the net input and/is the transfer function. (Reproduced from [26], by permission of John Wiley Sons, Ltd. copyright 2002.)...

Practically, the most widespread network diagram (basically used in the models described in this book) is a multilayer perceptron (MLP) , composed of many processing units (artificial neurons), each of which performs weighted summation of output signals, passing to the output channel their nonlinear function called activation function. The multilayer perceptron has only unidirectional connections between neurons of adjacent layers (no feedback, no coimections between neurons of the same layer, no connections between neurons of layers simated further than the directly adjacent ones) (Fig. 3.6). 

Artificial neural networks (ANN s) were effectively set aside for 15 years after a 1969 study by Minksy and Papert demonstrated their failure to correctly model a simple exclusive OR (XOR) function. i The XOR function describes the result of an operation involving two bits (1 or 0). A simple OR function produces a value of 1 if either bit or both bits have a value of 1. The XOR differs from an OR function in the output of an operation on two bits of value 1. The XOR function will yield a 0 while the OR function will yield a 1. Interest in ANN s resumed in the 1980 s after modifications were made to the layering of their neurons that allowed them to overcome the XOR test as well as a wide variety of other non-linear modeling challenges. 

Artificial neural networks (Hurtado 2004) are computational devices which permit the approximate calculation of outputs given an input set. The input is organized as a layer of neurons, each corresponding to one of the input variables, and the output is contained in an output layer. Intermediate, hidden layers, contain a number of neurons which receive information from the input layer and pass it on to subsequent layers. Each link in the network is associated with a weight w. The total information received by a neuron is processed by a transfer function h before being sent forward to the neurons in the next layer. For a network with a single hidden layer, the computational process can be expressed as... 

A multilayer perceptron (MLP) is a feed-forward artificial neural network model that maps sets of input data onto a set of suitable outputs (Patterson 1998). A MLP consists of multiple layers of nodes in a directed graph, with each layer fully connected to the next one. Except for the input nodes, each node is a neuron (or processing element) with a nonlinear activation function. MLP employs a supervised learning techruque called backpropagation for training the network. MLP is a modification of the standard linear perceptron and can differentiate data that are not linearly separable. 

---