The Hopfield Model

Chapter 10 covers another important field with a great overlap with CA neural networks. Beginning with a short historical survey of what is really an independent field, chapter 10 discusses the Hopfield model, stochastic nets, Boltzman machines, and multi-layered perceptrons. 

The Boltzman Machine generalizes the Hopfield model in two ways (1) like the simple stochastic variant discussed above, it t(>o substitutes a stochastic update rule for Hopfield s deterministic dynamics, and (2) it separates the neurons in the net into sets of visible and hidden units. Figure 10.8 shows a Boltzman Machine in which the visible neurons have been further subdividetl into input and output sets. 

An important aspect that did come forward in the 1970s was that of self-organizing maps (SOMs). SOMs will be discussed later in this encyclopedia. In 1982, John Hopfield of Caltech presented a paper to the scientific community in which he stated that the approach to AI should not be to purely imitate the human brain but instead to use its concepts to buUd machines that could solve dynamic problems. He showed what such networks were capable of and how they would work. It was his articulate, likable character and his vast knowledge of mathematical analysis that convinced scientists and researchers at the National Academy of Sciences to renew interest into the research of AI and neural networks. His ideas gave birth to a new class of neural networks that over time became known as the Hopfield model. 

A practical demonstration of this system is shown in Fig. 58 (without polarizers to avoid confusion), which shows the optical construction of a 64 neuron network with five stored memories [67]. The FLC SLMs were early 64 x 64 pixel devices developed at STC [68] two were used as the weight masks and the third was used to display the input data. The iteration time for the system was 50 ms and reconfiguration of the networks memory could be done in one cycle. The system was trained using the Hopfield model and used twelve vector sets each containing 10 memory vectors and two input... 

Fig. 35. Spin-state relaxation rate constant k versus temperature T for PSS-doped [Fe(6-Mepy)2(py)tren](CIOj2- Experimental data are indicated by filled circles. The solid line represents the fit to the tuimeling model of Hopfield, the dashed line the fit to the quantum mechanical theory of Buhks et al. According to Ref [138]...

This 3D lattice Hamiltonian with two traps can be elaborated further by adding filled lattice orbitals (3D bridge orbitals), which will then exhibit interference effects, and which could be computed by an extension of the Hopfield-Beratan method. Nevertheless the path counting given in Table 2 serves to illustrate the dramatic character of the difference between the single path model (Eq. 15) and the many path result (Eq. 14) in three dimensions. 

Figure 1. Schematic curves illustrate the dispersion relationship between the polariton frequency/co (k) and the wave-vector// . Figure 1(a) illustrates the dispersion if only a single molecular frequency is present (Hopfield model), and (b) the case of two molecular resonances (c) depicts a situation in which a number of such dispersion branches are present.

In the above treatment typically known as Hopfield model [11], in spite of correlations the memories are treated as independent. This is inconsistent. We argue that the very idea that a new memory should be lodged independently of (or without reference to) the previous memories is unrealistic and must be abandoned. For not only are the memories in general correlated, but the correlations are actually very much required to identify the fact that two objects, although different in some details, could belong to the same class of objects. So we must have a system that can deal with the correlations meaningfully and also circumvent the problem of noise arising out of them. 

Experiment 1 compares Hebbian trained Hopfield networks with their equivalent HEDA models. The aim is to discover whether or not the HEDA model can achieve the capacity of the Hopfield network. Hopfield networks were trained on patterns using standard Hebbian learning, with one pattern at a time being added until the network s capacity was exceeded. At this point, the learned patterns were set to be the targets for the HEDA search using Eqs. 21 and 22 and the network s weights were reset. 

More than 50 different types of neural network exist. Certain networks are more efficient in optimization others perform better in data modeling and so forth. According to Basheer (2000) the most popular neural networks today are the Hopfield networks, the Adaptive Resonance Theory (ART) networks, the Kohonen networks, the counter propagation networks, the Radial Basis Function (RBF) networks, the backpropagation networks and recurrent networks. 

Atrop is estimated with a mathematical model such as Hopfield model, modified Hopfield models, and Saastamoinen model. In these models, the tropospheric delay in zenith direction is estimated first and is multiplied by the mapping function which represents the effect of elevation angle. For more precise application, Atrop is treated as an unknown parameter. The ionospheric delay A,o also can be calculated with a mathematical model, but it is efficiently eliminated using the combination of LI and L2 frequency measurements (Hofmaim-Wellenhof et al. 1994). Therefore, L1/L2 GPS receiver is commonly used for determining the cmstal deformation. LI GPS receiver is used only in the case of short baseline. 

Feed-back models can be constructed and trained. In a constructed model, the weight matrix is created by adding the output product of every input pattern vector with itself or with an associated input. After construction, a partial or inaccurate input pattern can be presented to the network and, after a time, the network converges to one of the original input patterns. Hopfield and BAM are two well-known constructed feed-back models. 

Hopfield s neural net model addressed the basic associative memory problem [hopf82] Given some set of patterns Vi, construct a neural net such that when it is presented with an arbitrary pattern V, not necessarily an element of the given set, it responds with a pattern from the given set that most closely resembles" the presented pattern. 

Hopfield s model consists of a fnlly-coimected, symmetrically-weighted wij = Wji) McCulloch- Pitts neural net where the value of the neuron is updated according to ... 

The 3(X) — 4.2 K relaxation rate data for the PSS-doped complex were fitted to two different theoretical models. According to the model by Hopfield [172], electron tunneling between two molecular states is considered which are only very weakly interacting. The rate of tunneling between the two states a and b,... 

If we make a somewhat more realistic molecular model, such as that of linked springs (Hopfield),2 then just, as the volume of the sphere is dependent on a great variety of external effects, so must this structure respond at the molecular level to these external effects, but now all effects become directional. The idea of allosteric equilibrium (e.g., in hemoglo-... 

---