Associative memories

as mentioned at the close of the last section, it took more than 15 years following Minsky and Papert s criticism of simple perceptrons for a bona-fide multilayered variant to finally emerge (see Multi-layeved Perceptrons below), the man most responsible for bringing respectability back to neural net research was the physicist John J, Hopfield, with the publication of his landmark 1982 paper entitled Neural networks and physical systems with emergent collective computational abilities [hopf82]. To set the stage for our discussion of Hopfield nets, we first pause to introduce the notion of associative memory. 

Consider the basic probkun of how information is distributed throughout a system, and the manner in which it is retrieved. We. know that in a conventional computer, for example, information is stored in random-access memory (RAM). This means that the memory address of where the information actually exists and the information itself are uncorrelated. In order to retrieve the information, one must know its address exactly, as even the slightest error renders that information effectively unretrievable. In particular, it is in general impossible to retrieve RAM data if armed only with a partial knowledge of its address. In contrast, associative memories (sometimes also called content-addressable memories), much like the form of memory believed to be used by human brains, are such that they can be completely retrieved even when searched for with partial information, 

Hopfield s neural net model addressed the basic associative memory problem [hopf82] Given some set of patterns Vi, construct a neural net such that when it is presented with an arbitrary pattern V, not necessarily an element of the given set, it responds with a pattern from the given set that most closely resembles the presented pattern. 

While the solution to this problem can be trivially solved by a conventional serial computer program, of course by say computing the Hamming distance  

For each generated input pattern. Each unit has a bias set to be 

When an incoming pattern is processed the activity in postsynaptic unit is calculated as 

Here we sum over all groups and all units in each group where is the activation value of unit i. 

The final output is calculated by a softmax function, controlled by the gain parameter G, over all the units in the output layer  

In a classification task, the unit with the highest output is taken as the classification result. 

---

Kohonen, T. (1988) Self-Organization and Associative Memory, Springer-Verlag, Berlin. 

The second main category of neural networks is the feedforward type. In this type of network, the signals go in only one direction there are no loops in the system as shown in Fig. 3. The earliest neural network models were linear feed forward. In 1972, two simultaneous articles independently proposed the same model for an associative memory, the linear associator. J. A. Anderson [17], neurophysiologist, and Teuvo Kohonen [18], an electrical engineer, were unaware of each other s work. Today, the most commonly used neural networks are nonlinear feed-forward models. 

We now need to define the. set of symmetric weights, Wij wji. Given that we want our net to act as a content-associative memory, this set of weights must satisfy two conditions (1) if the net is presented with a pattern in the stored set, that set of encoded neuronal values representing that stored pattern should remain unclianged - i.e. stored patterns must be stable (2) if the net is presented with a pattern not in the stored set, the net should evolve according to equation 10.7 to the pattern that is closest to it. 

Kohonen T. Self-organisation and associative memory. Berlin Springer-Verlag, 1989. 

HIV produces profound CD4 depletion, possibly through initial massive depletion of gut-associated memory T-cells, followed by chronic immune activation, leading to fatigue of homeostatic T-cell responses and progressive immunodeficiency (McArthur et al. 2005). 

T. Kohonen, Self Organization and Associated Memory. Springer-Verlag, Heidelberg, 1989. W.J. Meissen, J.R.M. Smits, L.M.C. Buydens and G. Kateman, Using artificial neural networks for solving chemical problems. II. Kohonen self-organizing feature maps and Hopfield networks. Chemom. Intell. Lab. Syst., 23 (1994) 267-291. 

Both cases can be dealt with both by supervised and unsupervised variants of networks. The architecture and the training of supervised networks for spectra interpretation is similar to that used for calibration. The input vector consists in a set of spectral features yt(Zj) (e.g., intensities at selected wavelengths zi). The output vector contains information on the presence and absence of certain structure elements and groups fixed by learning rules (Fig. 8.24). Various types of ANN models may be used for spectra interpretation, viz mainly such as Adaptive Bidirectional Associative Memory (BAM) and Backpropagation Networks (BPN). The correlation... 

Kohonen T (1984) Self-organization and associative memory. Springer, Berlin Heidelberg New York... 

Data from the few available case reports and the numerous animal studies strongly suggest that disulfoton exposure may cause mild to severe neurological effects in humans. Although disulfoton-associated memory loss has not been reported in humans, there should be concern that occupational exposure and exposure at hazardous waste sites may interfere with the performance of cognitive and complex tasks. 

Semlitsch HV, Anderer P, Saletu B, Binder GA, Decker KA. (1995). Cognitive psychophysiology in nootropic drug research effects of Ginkgo biloba on event-related potentials (P300) in age-associated memory impairment. Pharmacopsychiatry. 28(4) 134-42. 

ASSOCIATIVE DATA BASE MANAGER ASSOCIATIVE MEMORY... 

In a study of age-associated memory impairment [Crook and Lakin 1991), using a double-blind, placebo-controlled trial and three doses of ondansetron (10, 250, or 1,000 pg po bid), patients were treated for 12 weeks, followed by a 2-week washout period, with assessments being made at the initiation of, during, and at the termination of treatment. Behavioral rating scales and a computerized battery of tests related to learning and memory tasks of daily life were used to assess changes in cognitive performance. Ondansetron caused dose-related effects to enhance acquisition, name-face associa-... 

Crook TH, Lakin M Effects of ondansetron image-associated memory impairment, in Biological Psychiatry Proceedings of the 5th World Congress of Biological Psychiatry, Florence, 9-14 June 1991. Edited by Racagni G, Brunello N, Fukuda T. New York, Excerpta Medica, 1991, pp 1-4... 

Crook T, Bartus RT, Ferris St. H, et al Age-associated memory impairment proposed diagnostic criteria and measures of clinical change— report of a National Institute of Health Work Group. Developmental Neuropsychology 2(4 261-276,... 

Schierle and Otto [63] used a two-layer perceptron with error back-propagation for quantitative analysis in ICP-AES. Also, Schierle et al. [64] used a simple neural network [the bidirectional associative memory (BAM)] for qualitative and semiquantitative analysis in ICP-AES. 

Z. Zhang, H. Zhuo, S. Liu and P. D. B. Harrington, Classification of cancer patients based on elemental contents of serums using bidirectional associative memory networks. Anal. Chim. Acta, 436(2), 2001, 281-291. 

T. Kohonen, S ef-Organisation and Associative Memory, Springer-VerLag, Berlin, 1987. 

Wagstaff LR, Mitton MW, Arvik BM, Doraiswamy PM. Statin-associated memory loss analysis of 60 case reports and review of the literature. Pharmacotherapy 1920 23 871-80. 

---