Analysis of neural networks

We said previously that a shortcoming of ANNs is the difficulty of determining how a network makes its decisions. While this is a difficult task, it is certainly not impossible to solve, and now we turn to a description of some ways of accomplishing this. 

Another approach to weight analysis is to consider just the weights as forming a vector. Consider a three-layer network (input, hidden, and output). If all the network weights are considered to be components of a vector W, then this vector will be of dimension N = ( i h + where j, i, and are the numbers of input, hidden, and output PEs. [If there is a bias connected to the hidden and output PEs, N = (wj + l) h + ( [, + l) o ] If we define an N-dimensional unit vector as S =. . ., 1), then the angle W makes 

To ascertain what rules an ANN learns, one can resort to some form of sensitivity analysis. In its most general form, sensitivity analysis addresses the following question For a given change in an input or inputs, what is the corresponding change in an output or outputs Sensitivity analysis can be utilized in several contexts, one of which is pruning. 

Aoyama and Ichikawa have given analytic formulas for the partial derivatives of the output value of either a HL or output layer PE with respect to the input value of an input PE. Their formulas, which are applicable to any feedforward network with differentiable transfer functions in the hidden and output layers, allow you to give a precise, analytical answer to the question that sensitivity analysis asks (see above). A similar sensitivity analysis has been performed for a radial basis function ANN. Aoyama and coworkers introduced another technique useful in network analysis the reconstruction of weight matrices for a backpropagation network. They used a learning 

---

Relating measurements. Evaluating the relationships between the different types of measurements of the variables that are coupled or not to a process is fundamental in statistics. In the case of variables coupled to a process, the separation in the class of independent variables (xj, I = 1, n) and dependent variables (yj, j = 1, p) must be established based on the schematic representation of the process (see Fig. 1.1 in Chapter 1). The statistical models will be built based on experimental measurements. However, good models can be developed only if experimental results are obtained and processed from a statistical analysis. The analysis of neural networks processes, which are also statistical models, represents a modern and efficient research techmque based on the experimental measurement of one actual process. 

Berry, M. and Flinn, R. (1982) Vertex analysis of neural networks, J. Comp. Neurol In press. 

Woodruff and co-workers introduced the expert system PAIRS [67], a program that is able to analyze IR spectra in the same manner as a spectroscopist would. Chalmers and co-workers [68] used an approach for automated interpretation of Fourier Transform Raman spectra of complex polymers. Andreev and Argirov developed the expert system EXPIRS [69] for the interpretation of IR spectra. EXPIRS provides a hierarchical organization of the characteristic groups that are recognized by peak detection in discrete ames. Penchev et al. [70] recently introduced a computer system that performs searches in spectral libraries and systematic analysis of mixture spectra. It is able to classify IR spectra with the aid of linear discriminant analysis, artificial neural networks, and the method of fe-nearest neighbors. 

Applications of neural networks are becoming more diverse in chemistry [31-40]. Some typical applications include predicting chemical reactivity, acid strength in oxides, protein structure determination, quantitative structure property relationship (QSPR), fluid property relationships, classification of molecular spectra, group contribution, spectroscopy analysis, etc. The results reported in these areas are very encouraging and are demonstrative of the wide spectrum of applications and interest in this area. 

Since that time thousands of QSARs, covering a wide and diverse range of end points, have been published [9] most of these have used MLR, but numerous other statistical techniques have also been used, such as partial least squares, principal component analysis, artificial neural networks, decision trees, and discriminant analysis [f4]. 

Two models of practical interest using quantum chemical parameters were developed by Clark et al. [26, 27]. Both studies were based on 1085 molecules and 36 descriptors calculated with the AMI method following structure optimization and electron density calculation. An initial set of descriptors was selected with a multiple linear regression model and further optimized by trial-and-error variation. The second study calculated a standard error of 0.56 for 1085 compounds and it also estimated the reliability of neural network prediction by analysis of the standard deviation error for an ensemble of 11 networks trained on different randomly selected subsets of the initial training set [27]. 

Non-linear PCA can be obtained in many different ways. Some methods make use of higher order terms of the data (e.g. squares, cross-products), non-linear transformations (e.g. logarithms), metrics that differ from the usual Euclidean one (e.g. city-block distance) or specialized applications of neural networks [50]. The objective of these methods is to increase the amount of variance in the data that is explained by the first two or three components of the analysis. We only provide a brief outline of the various approaches, with the exception of neural networks for which the reader is referred to Chapter 44. 

By design, ANNs are inherently flexible (can map nonlinear relationships). They produce models well suited for classification of diverse bacteria. Examples of pattern analysis using ANNs for biochemical analysis by PyMS can be traced back to the early 1990s.4fM7 In order to better demonstrate the power of neural network analysis for pathogen ID, a brief background of artificial neural network principles is provided. In particular, backpropagation artificial neural network (backprop ANN) principles are discussed, since that is the most commonly used type of ANN. 

O. D. Sanni, M. S. Wagner, D. G. Briggs, D. G. Castner and J. C. Vickerman, Classification of adsorbed protein static ToF SIMS spectra by principal component analysis and neural networks, Surface and Interface Analysis, 33, 715 728 (2002). 

Another way for BOD estimation is the use of sensor arrays [37]. An electronic nose incorporating a non-specific sensor array of 12 conducting polymers was evaluated for its ability to monitor wastewater samples. A statistical approach (canonical correlation analysis) showed a linear relationship between the sensor responses and BOD over 5 months for some subsets of samples, leading to the prediction of BOD values from electronic nose analysis using neural network analysis. 

HT(la) /alpha( 1)-adrenergic receptor affinity by classical Hansch analysis, artificial neural networks, and computational simulation of ligand recognition. Journal of Medicinal Chemistry, 44, 198-207. 

Lopez-Rodriguez, M.L., Morcillo, M.J., Fernandez, E., Rosado, M.L., Pardo, L. and Schaper, K.-f. (2001) Synthesis and structure-activity relationships of a new model of arylpiperazines. 6. Study of the 5-HTiA/ai-adrenergic receptor affinity by classical Hansch analysis, artificial neural networks, and computational simulation of ligand recognition. Journal of Medicinal Chemistry, 44, 198-207. 

Another QSAR study utilizing 14 flavonoid derivatives in the training set and 5 flavonoid derivatives in the test set was performed by Moon et al. (211) using both multiple linear regression analysis and neural networks. Both statistical methods identified that the Hammett constant a, the HOMO energy, the non-overlap steric volume, the partial charge of C3 carbon atom, and the HOMO -coefficient of C3, C3, and C4 carbon atoms of flavonoids play an important role in inhibitory activity (Eqs. 3-5, Table 5). 

Edin, B.B. and Trulsson, M.(1992) Neural network analysis of the information content in population responses from human periodontal receptors. Science of Neural Networks SPIE 1710 257-266... 

Nykamp D. Q., Trandrina D. A population density approach that facilitates large-scale modeling of neural networks analysis and an application to orientation tuning. J Comput Neurosci, 2000, 8(1), 19-50. 

---