Training the Network

ANN to memorize that case. The test set should also contain a representative sampling of cases to realistically assess how the ANN responds to new situations. A word about autoassociation problems is in order here. If your goal is to use an ANN to simply store patterns or to compress data, you really do not need a test set because all you care about are the cases with which you train the network. If you want to pass corrupt data through the ANN to see if the network will output a clean version of the input, you may want to construct a test set to see how well the network can do this training set construction is presumably trivial here you know what data you want to store or compress, and this data is the training set. 

You are now ready to set up and train the network. To set up the network, you need to select the number of layers of PEs, the number of PEs in each layer, and the values for any adjustable parameters (learning rates, etc.). For many ANNs the architecture question is almost trivial there is a well-defined number of layers and a well-defined number of PEs in each layer. In all cases, however. 

Most ANNs have a required number of layers, each performing a specific role. The backpropagation network allows some flexibility in the number of hidden layers. Generally you should use one HL, although it is possible that two HLs will perform better if the total number of network PEs is less than for the one HL cases. Often, the best way to decide how many HLs to use is simply to experiment. 

Most ANNs have specific transfer functions that must be used in a given layer. Once again, the backpropagation network is an exception. Whereas the 

At this stage, you are ready to train the network. If you are using a backpropagation network, you should randomize the order of cases in the training set and change the order after every few epochs. This Mali help counter the tendency of these networks to learn the first cases better than later ones. We suggest that you use this randomization scheme no matter what network you 

---

The resulting distribution of compounds having different modes of action in the output layer after training the network is shown in figure 10.1-10. 

The abihty to generalize on given data is one of the most important performance charac teristics. With appropriate selection of training examples, an optimal network architec ture, and appropriate training, the network can map a relationship between input and output that is complete but bounded by the coverage of the training data. 

P is a vector of inputs and T a vector of target (desired) values. The command newff creates the feed-forward network, defines the activation functions and the training method. The default is Fevenberg-Marquardt back-propagation training since it is fast, but it does require a lot of memory. The train command trains the network, and in this case, the network is trained for 50 epochs. The results before and after training are plotted. 

Finally, any training is incomplete without proper validation of the trained model. Therefore, the trained network should be tested with data that it has not seen during the training. This procedure was followed in this study by first training the network on one data set, and then testing it on a second different data set. 

Figure 2.28 shows how the sum over all samples of the Euclidean distance, E between target output and actual output varies as a function of the number of epochs during a typical training session. Ed for the training set (solid line) falls continuously as training occurs, but this does not mean that we should train the network for as long as we can. 

This method for preventing overfitting requires that there are enough samples so that both training and test sets are representative of the dataset. In fact, it is desirable to have a third set known as a validation set, which acts as a secondary test of the quality of the network. The reason is that, although the test set is not used to train the network, it is nevertheless used to determine at what point training is stopped, so to this extent the form of the trained network is not completely independent of the test set. 

A second sample is then chosen from the dataset and fed through the network once again the network output is compared with the desired output and the network weights are adjusted in order to reduce the difference between desired and actual output. This process is repeated until all samples in the dataset have been fed through the network once this constitutes one epoch. Many epochs are normally required to train the network, especially if the dataset is both small and diverse. In an alternative, and broadly equivalent, procedure the updating of the weights only occurs after the complete set of samples has been observed by the network, that is, after each epoch. 

The major advantage of the prepared neural network FCC model is that it does not require a lot of input information. In addition, the model can be updated whenever new input-output information for the FCC unit is made available. This can be done by retraining the neural network starting from the old connection weights as an initial guess for the optimization process and by including the new set of data within the overall set used to train the network. 

Before training the Network, it is necessary to check the data in order to avoid conflicts such as having maximums and minimums among internal validation or test samples. To do so, click /Action/Check Grid and press Yes in the new window that appears. 

In the styrene—butadiene copolymer application, a series of quantitative ANN models for the as-butadiene content was developed. For each of these models, all of the 141 X-variables were used as inputs, and the sigmoid function (Equation 8.39) was used as the transfer function in the hidden layer. The X-data and Y-data were both mean-centered before being used to train the networks. A total of six different models were built, using one to six nodes in the hidden layer. The model fit results are shown in Table 8.7. Based on these results, it appears that only three, or perhaps four, hidden nodes are required in the model, and the addition of more hidden nodes does not greatly improve the fit of the model. Also, note that the model fit (RMSEE) is slightly less for the ANN model that uses three hidden nodes (1.13) than for the PLS model that uses four latent variables (1.25). 

The error between the actual mismatch (obtained from the simulation results) and that predicted by the network is used as the error signal to train the network (see Figure 12.3). This is a classical supervised learning problem, where the system provides target values directly to the output co-ordinate system of the learning network. 

The four experiments done previously with Rnp (= 0.5, 1, 3, 4) were used to train the neural network and the experiment with / exp = 2 was used to validate the system. Dynamic models of process-model mismatches for three state variables (i.e. X) of the system are considered here. They are the instant distillate composition (xD), accumulated distillate composition (xa) and the amount of distillate (Ha). The inputs and outputs of the network are as in Figure 12.2. A multilayered feed forward network, which is trained with the back propagation method using a momentum term as well as an adaptive learning rate to speed up the rate of convergence, is used in this work. The error between the actual mismatch (obtained from simulation and experiments) and that predicted by the network is used as the error signal to train the network as described earlier. 

The only difficult part is finding the values for p. and o for each hidden unit, and the weights between the hidden and output layers, Le., training the network. This will be discussed later, in Chapter 5. At this point, it is sufficient to say that training radial basis function networks is considerably faster than training multilayer perceptrons. On the other hand, once trained, the feed-forward process for multilayer perceptrons is faster than for radial basis function networks. 

Training the network is nothing more than an optimization problem. When a large data set is used, or when the network has a large number of weights and biases, this task can... 

Once a neural network is trained — that is, its weights are adapted to the input data — we can present a new input vector and predict a value or vector in the output layer. Although the training procedure can be quite time consuming, once trained the network produces an answer (i.e., prediction) almost instantaneously. A more detailed discussion of applications of neural networks in chemistry and drug design can be found in Zupan and Gasteiger [60]. 

A CPG neural network can help to find the appropriate interelement coefficients by training the network with pairs of descriptors, one of which contains the raw count rates for the interfering elements whereas the other contains experimentally determined interelement coefficients (Figure 6.29). Training the network with values of a well-defined type of chemical matrix, the network is able to predict the interelement coefficients that can finally be used to correct the calibration graph used for determining the element concentrations (Figure 6.30). 

FIGURE 6.29 Schematic view of a CPG neural network trained with vector pairs. The input vector consists of ten countrates for major elements in rock samples, whereas the output vector contains the interelement coefficients p for all major elements. After training, the network is able to predict the interelement coefficients for the given sample matrix. 

---