Training a network

Most recent scientific applications involve the determination of direct relationships between input parameters and a known target response. For example, Santana and co-workers have used ANNs to relate the structure of a hydrocarbon to its cetane number,4 while Berdnik s group used a theoretical model of light scattering to train a network that was then tested on flow cytometry data.5... 

A major criticism of neural networks is that it is difficult to interpret the meaning of the weights, after training. A network s performance may be acceptable but lend no insight into how it works, a problem reminiscent of principal component analysis component functions are often hard to interpret. This is a difficult and important challenge for future research. 

The next major development in neural network technology arrived in 1949 with a book. The Organization of Behavior written by Donald Hebb. The book supported and further reinforced McCulloch-Pitts s theory about neurons and how they work. A major point bought forward in the book described how neural pathways are strengthened each time they were used. As we shall see, this is trae of neural networks, specifically in training a network. 

Given a function to implement as a neural network, there are a number of (currently) ill-understood and unconventional stages in the process. Instead of designing an algorithm to meet the specification, we train a network to do so, and in both cases we test the implementation to determine whether it is acceptable. Currently, we are using Multilayer Perceptron nets trained with the backpropagation algorithm [8]. 

In anticipation of the introduction of risk management requirements under 1EC80001 (or in case of expansion of the scope of the Medical Devices Directive), the NHS Connecting for Health (CfH) Chnical Safety Group commenced creating and training a network of Trust based Clinical Safety Officers to assist in the safety management tasks associated with new systems deployment. 

There are finer details to be extracted from such Kohonen maps that directly reflect chemical information, and have chemical significance. A more extensive discussion of the chemical implications of the mapping of the entire dataset can be found in the original publication [28]. Gearly, such a map can now be used for the assignment of a reaction to a certain reaction type. Calculating the physicochemical descriptors of a reaction allows it to be input into this trained Kohonen network. If this reaction is mapped, say, in the area of Friedel-Crafts reactions, it can safely be classified as a feasible Friedel-Qafts reaction. 

The profits from using this approach are dear. Any neural network applied as a mapping device between independent variables and responses requires more computational time and resources than PCR or PLS. Therefore, an increase in the dimensionality of the input (characteristic) vector results in a significant increase in computation time. As our observations have shown, the same is not the case with PLS. Therefore, SVD as a data transformation technique enables one to apply as many molecular descriptors as are at one s disposal, but finally to use latent variables as an input vector of much lower dimensionality for training neural networks. Again, SVD concentrates most of the relevant information (very often about 95 %) in a few initial columns of die scores matrix. 

Steps 2 and 3 are performed for all input objects. When all data points have been fed into the network one training epoch has been achieved. A network is usually trained in several training epochs, depending on the size of the network and the number of data points. 

The trained counterpvopagation network is then able to predict the spectrum for a new structure when operating as a look-up table (see Figure 10.2-9) the encoded query or th.c input structure is input into the trained network and the winning neuron is determined by considering just the upper part of the network. The neuron points to the corresponding neuron in the lower part of the network, which then provides the simulated IR spectrum. 

Next wc turned our attention to the question of whether wc could still sec the separation of the two sets of molecules when they were buried in a large data set of diverse structures. For this purpose we added this data set of 172 molecules to the entire catalog of 8223 compounds available from a chemical supplier (janssen Chimica). Now, having a larger data set one also has to increase the size of the network a network of 40 X 30 neurons was chosen. Training this network with the same 49-dimcnsional structure representation as previously described, but now for all 8395 structures, provided the map shown in Figure 10,4-9. 

The abihty to generalize on given data is one of the most important performance charac teristics. With appropriate selection of training examples, an optimal network architec ture, and appropriate training, the network can map a relationship between input and output that is complete but bounded by the coverage of the training data. 

All elosed-loop eontrol systems operate by measuring the error between desired inputs and aetual outputs. This does not, in itself, generate eontrol aetion errors that may be baek-propagated to train a neural network eontroller. If, however, a neural network of the plant exists, baek-propagation through this network of the system error (r(lcT) — y(lcT)) will provide the neeessary eontrol aetion errors to train the neural network eontroller as shown in Figure 10.29. 

P is a vector of inputs and T a vector of target (desired) values. The command newff creates the feed-forward network, defines the activation functions and the training method. The default is Fevenberg-Marquardt back-propagation training since it is fast, but it does require a lot of memory. The train command trains the network, and in this case, the network is trained for 50 epochs. The results before and after training are plotted. 

The number of neurons to be used in the input/output layer are based on the number of input/output variables to be considered in the model. However, no algorithms are available for selecting a network structure or the number of hidden nodes. Zurada [16] has discussed several heuristic based techniques for this purpose. One hidden layer is more than sufficient for most problems. The number of neurons in the hidden layer neuron was selected by a trial-and-error procedure by monitoring the sum-of-squared error progression of the validation data set used during training. Details about this proce-... 

Finally, any training is incomplete without proper validation of the trained model. Therefore, the trained network should be tested with data that it has not seen during the training. This procedure was followed in this study by first training the network on one data set, and then testing it on a second different data set. 

Even so, artificial neural networks exhibit many brainlike characteristics. For example, during training, neural networks may construct an internal mapping/ model of an external system. Thus, they are assumed to make sense of the problems that they are presented. As with any construction of a robust internal model, the external system presented to the network must contain meaningful information. In general the following anthropomorphic perspectives can be maintained while preparing the data ... 

A network that is too large may require a large number of training patterns in order to avoid memorization and training time, while one that is too small may not train to an acceptable tolerance. Cybenko [30] has shown that one hidden layer with homogenous sigmoidal output functions is sufficient to form an arbitrary close approximation to any decisions boundaries for the outputs. They are also shown to be sufficient for any continuous nonlinear mappings. In practice, one hidden layer was found to be sufficient to solve most problems for the cases considered in this chapter. If discontinuities in the approximated functions are encountered, then more than one hidden layer is necessary. 

---