McCulloch

Artificial Neural Networks (ANNs) attempt to emulate their biological counterparts. McCulloch and Pitts (1943) proposed a simple model of a neuron, and Hebb (1949) described a technique which became known as Hebbian learning. Rosenblatt (1961), devised a single layer of neurons, called a Perceptron, that was used for optical pattern recognition. 

McCulloch, W.S. and Pitts, W.H. (1943) A logical calculus of ideas immanent in nervous activity. Bulletin of Mathematical Biophysics, 5, pp. 115-133. 

Gattuso M. J., McCulloch B., House D. W., Baumann W. M., Gottschall K. (1996) Simulated Moving Bed Technology - The Preparation of Single Enantiomer Drugs, Pharm. Tech. Europe 8 20-25. 

First Neuron Model McCulloch-Pitts [mccul43]... 

Fig. 10.1 Schematic representations of (a) a biological neuron, and (b) a McCulloch-Pitts artificial neuron.

Now, to be sure, McCulloch-Pitts neurons are unrealistically rendered versions of the real thing. For example, the assumption that neuronal firing occurs synchronously throughout the net at well defined discrete points in time is simply wrong. The tacit assumption that the structure of a neural net (i.e. its connectivity, as defined by the set of synaptic weights) remains constant over time is known be false as well. Moreover, while the input-output relationship for real neurons is nonlinear, real neurons are not the simple threshold devices the McCulloch-Pitts model assumes them to be. In fact, the output of a real neuron depends on its weighted input in a nonlinear but continuous manner. Despite their conceptual drawbacks, however, McCulloch-Pitts neurons are nontrivial devices. McCulloch-Pitts were able to show that for a suitably chosen set of synaptic weights wij, a synchronous net of their model neurons is capable of universal computation. This means that, in principle, McCulloch-Pitts nets possess the same raw computational power as a conventional computer (see section 6.4). 

Hopfield s model consists of a fnlly-coimected, symmetrically-weighted wij = Wji) McCulloch- Pitts neural net where the value of the neuron is updated according to ... 

Consider the Boolean exclusive-OR (or XOR) function that we used as an example of a linearly inseparable problem in our discussion of simple perceptrons. In section 10.5.2 we saw that if a perceptron is limited to having only input and output layers (and no hidden layers), and is composed of binary threshold McCulloch-Pitts neurons, the value y of its lone output neuron is given by... 

This memory erasure problem is sometimes called the credit assignment problem [peret92l. Fortunately, there is an easy way out. We merely generalize the binary (on/off) McCulloch-Pitts neuronal values to continuous variables by smoothing out the step-function threshold. 

As we will discuss a bit later on in this section, adding hidden layers but generalizing the McCulloch-Pitts step-function thresholding to a linear function yields a multi-layer perceptron that is fundamentally no more powerful than a simple perceptron that has no hidden layers and uses the McCulloch-Pitts step-function threshold. 

Simple Perceptrons In the case of simple perceptroiis using McCulloch-Pitts thresholding, the answer depends on a careful analysis of linear separability. 

Ellis, W. G. Schneider E. L. McCulloch, J. R. Suzuki R. and Epstein, C. J. "Fetal Globoid Cell Leukodystrophy (Rrabbe s Disease) Pathological and Biochemical Examination". Arch. Neurol., (1973), 29, 253-257. 

W.S. McCulloch and W. Pitts, A logical calculus of the ideas immanent in nervous activity. Bull. Math. Biophy., 5 (1943) 115-133. 

Wasserburg GJ, Jacobsen SB, DePaolo DJ, McCulloch MT, and Wen T (1981) Precise determination of Sm/Nd ratios, Sm and Nd isotopic abundances in standard solutions. Geochim Cosmochim Acta 45 2311-2323... 

---