McCulloch and Pitts

to be sure, McCulloch-Pitts neurons are unrealistically rendered versions of the real thing. For example, the assumption that neuronal firing occurs synchronously throughout the net at well defined discrete points in time is simply wrong. The tacit assumption that the structure of a neural net (i.e. its connectivity, as defined by the set of synaptic weights) remains constant over time is known be false as well. Moreover, while the input-output relationship for real neurons is nonlinear, real neurons are not the simple threshold devices the McCulloch-Pitts model assumes them to be. In fact, the output of a real neuron depends on its weighted input in a nonlinear but continuous manner. Despite their conceptual drawbacks, however, McCulloch-Pitts neurons are nontrivial devices. McCulloch-Pitts were able to show that for a suitably chosen set of synaptic weights wij, a synchronous net of their model neurons is capable of universal computation. This means that, in principle, McCulloch-Pitts nets possess the same raw computational power as a conventional computer (see section 6.4). 

---

Artificial Neural Networks (ANNs) attempt to emulate their biological counterparts. McCulloch and Pitts (1943) proposed a simple model of a neuron, and Hebb (1949) described a technique which became known as Hebbian learning. Rosenblatt (1961), devised a single layer of neurons, called a Perceptron, that was used for optical pattern recognition. 

There has been a steady development of neuronal analogs over the past 50 years. An important early model was proposed in 1943 by McCulloch and Pitts [23]. They described the neuron as a logical processing unit, and the influence of their model set the mathematical tone of what is being done today. Adaption or learning is a major focus of neural net research. The development of a learning rule that could be used for neural models was pioneered by Hebb, who proposed the famous Hebbian model for synaptic modification [24]. Since then, many alternative quantitative interpretations of synaptic modification have been developed [15-22]. 

Neural networks are essentially non-linear regression models based on a binary threshold unit (McCulloch and Pitts, 1943). The structure of neural networks, called a perception, consists of a set of nodes at different layers where the node of a layer is linked with all the nodes of the next layer (Rosenblatt, 1962). The role of the input layer is to feed input patterns to intermediate layers (also called hidden layers) of units that are followed by an output result layer where the result of computation is read-off. Each one of these units is a neuron that computes a weighted sum of its inputs from other neurons at a previous layer, and outputs a one or a zero according to whether the sum is above or below a... 

Neural networks are algorithmic systems introduced by McCulloch and Pitts in 1943 [24]. Their main advantage, compared with deterministic models, is their ability to provide information about the system to be modeled without prior knowledge of the physical process and identification of the mechanisms involved. Another important advantage of neural networks, compared to the data-based models like regressions or polynomials, is that they are non-linear parsimonious approximators, involving modeling of non-linear industrial processes with a rninimum number of parameters [25]. 

To address general questions of connectivity and function, the system is modeled with time-discrete, binary neurons (McCulloch and Pitts 1943) which fire (have value 1) whenever the sum of active, connected neurons in the previous time step is larger than their firing threshold 6,... 

The history of NNs can be traced back to 1943, when physiologists McCulloch and Pitts established the model of a neiuon as a binary linear threshold rmit (McCulloch and Pitts, 1943). One of the most well-known features of NNs is that they can be used as universal approximators (Scarselli and Tsoi, 1998 Zhang et al., 2012). In view of this feature, NNs have been widely applied to a variety of related problems, such as forecasting, modeling, classification and clustering. 

The origins of artificial neural networks (ANN) date back to the 1940s when W. McCulloch and W. Pitts presented the first simple systems (McCulloch and Pitts, 1945). Their basic idea is depicted in Fig. 14.10. Data sets are fed into the input layer and passed on to one or more hidden layers where the information is processed, thus generating knowledge before the output is calculated. 

Fortunately, ANNs can overcome these limitations and be used to develop models for these types of data. Some of the earliest work with neural networks was done by McCulloch and Pitts in 1943. ANNs can be used for the evaluation of nonlinear data for the development of a predictive model. Thus, a nonlinear data set, such as the class system of CPT data in the USDA archive, can be used to develop a model and predict compound activities based on the compound structures and associated repellent activities that were incorporated into the neural network. Three-layer neural networks with different architectures were applied to the data sets of acylpiperidines in this chapter. 

---