Associative memory networks

Z. Zhang, H. Zhuo, S. Liu and P. D. B. Harrington, Classification of cancer patients based on elemental contents of serums using bidirectional associative memory networks. Anal. Chim. Acta, 436(2), 2001, 281-291. 

Associative memory networks A type of recurrent network whose equilibrium state is used to memorize information. 

Two-layer feedforward/feedback ANNs are heteroassociative. They can store input and output vectors and are useful in recalling an output vector when presented with a noisy or incomplete version of its corresponding input vector. They are also useful for classification problems. Typically, every feedforward connection between two PEs is accompanied by a feedback connection between the same two PEs. Both connections have weights, and these weights are usually different from each other. Examples are the adaptive resonance theory and bidirectional associative memory networks. 

Pattern classification and associative memory networks can be trained to distinguish patterns into separate classes and associate input-output pairs. 

The second main category of neural networks is the feedforward type. In this type of network, the signals go in only one direction there are no loops in the system as shown in Fig. 3. The earliest neural network models were linear feed forward. In 1972, two simultaneous articles independently proposed the same model for an associative memory, the linear associator. J. A. Anderson [17], neurophysiologist, and Teuvo Kohonen [18], an electrical engineer, were unaware of each other s work. Today, the most commonly used neural networks are nonlinear feed-forward models. 

While, as mentioned at the close of the last section, it took more than 15 years following Minsky and Papert s criticism of simple perceptrons for a bona-fide multilayered variant to finally emerge (see Multi-layeved Perceptrons below), the man most responsible for bringing respectability back to neural net research was the physicist John J, Hopfield, with the publication of his landmark 1982 paper entitled Neural networks and physical systems with emergent collective computational abilities [hopf82]. To set the stage for our discussion of Hopfield nets, we first pause to introduce the notion of associative memory. 

T. Kohonen, Self Organization and Associated Memory. Springer-Verlag, Heidelberg, 1989. W.J. Meissen, J.R.M. Smits, L.M.C. Buydens and G. Kateman, Using artificial neural networks for solving chemical problems. II. Kohonen self-organizing feature maps and Hopfield networks. Chemom. Intell. Lab. Syst., 23 (1994) 267-291. 

Both cases can be dealt with both by supervised and unsupervised variants of networks. The architecture and the training of supervised networks for spectra interpretation is similar to that used for calibration. The input vector consists in a set of spectral features yt(Zj) (e.g., intensities at selected wavelengths zi). The output vector contains information on the presence and absence of certain structure elements and groups fixed by learning rules (Fig. 8.24). Various types of ANN models may be used for spectra interpretation, viz mainly such as Adaptive Bidirectional Associative Memory (BAM) and Backpropagation Networks (BPN). The correlation... 

Schierle and Otto [63] used a two-layer perceptron with error back-propagation for quantitative analysis in ICP-AES. Also, Schierle et al. [64] used a simple neural network [the bidirectional associative memory (BAM)] for qualitative and semiquantitative analysis in ICP-AES. 

Some historically important artificial neural networks are Hopfield Networks, Per-ceptron Networks and Adaline Networks, while the most well-known are Backpropa-gation Artificial Neural Networks (BP-ANN), Kohonen Networks (K-ANN, or Self-Organizing Maps, SOM), Radial Basis Function Networks (RBFN), Probabilistic Neural Networks (PNN), Generalized Regression Neural Networks (GRNN), Learning Vector Quantization Networks (LVQ), and Adaptive Bidirectional Associative Memory (ABAM). 

S. Bicciato M. Pandin G. Didone C. D. Bello, In Analysis of an Associative Memory Neural Network for Pattern Identification in Gene Expression Data, Proceedings of 1st Workshop on Data Mining in Bioinformatics, M. J. Zaki, H. T. T. Toivonen, J. T. L. Wang, Eds. San Francisco, CA, USA, pp 22-30. 

The severe memory loss of AD is associated with decreased neuronal activity in the medial temporal lobe memory network, which includes the hippocampus. The decline in long-term memory is most typical of Alzheimer s disease. Elderly persons with no symptoms of dementia may have difficulty concentrating on one thing when distractions are present. They become confused in complex, novel situations (Buckner, 2004). 

We have previously developed and investigated biophysically detailed models of the associative memory function of neocortex based on experimental data (Lundqvist et al. 2006). Based on the knowledge gained we have formulated an abstract network model of cortical layers 2/3 that forms the core of our present approach (Lansner and Holst 1996 Sandberg et al. 2002 Johansson and Lansner 2006a). Layer 5 is also likely to be closely interacting with layers 2/3 and is not represented separately (Hirsch and Martinez 2006). 

The bidirectional associative memory (BAM) is used here to explain the operation of a neural network in more detail. 

Carpenter, G. 1989. Neural network models for pattern recognition and associative memory. Neural Networks, 2 243-258. 

Kubota, T. A higher order associative memory with MccuUoch-Pitts neurons and plastic synapses. In International Joint Conference on Neural Networks, IJCNN 2007, pp. 1982-1989 (2007)... 

B. Lenze, Neural Networks, 11, 1041 (1998). Complexity Preserving Increase of the Capacity of Bidirectional Associative Memories by Dilation and Translation. 

The concept of the autoassociative memory was extended to bidirectional associative memories (BAM) by Kosko (1987,1988). This memory, shown in Fig. 19.30, is able to associate pairs of the patterns a and b. This is the two-layer network with the output of the second layer connected directly to the input of the first layer. The weight matrix of the second layer is and W for the first layer. The rectangular weight matrix W is obtained as a sum of the cross-correlation matrixes... 

Matrix modulators or controlled transparencies could be used in data-processing systems for correction of the optical aberration in real-time images [49], as well as writing and reconstructing the holographic information [49, 50], which is a basic tool for optical pattern classification [51], modeling of neural networks [52], optical associative memory [53], phase conjugation of low-power optical beams [54], etc. 

There is nothing like the neural network . Many types of neural networks have been proposed and used for various purposes, like Hopfield networks,the adaptive bidirectional associative memory,the Kohonen network, and radial basis funetion For the con-... 

ANN = artificial neural network AR = approximate reasoning BAM = bidirectional associative memory FAM = fuzzy associative memory FL = fuzzy logic LP = linear programming OLS = ordinary least squares PCCF = partial cross correlation function SSC = spectrum-structure correlation sup = supremum = fuzzy addition = fuzzy subtraction O = fuzzy multiplication Q = fuzzy division Q = relation. 

A summary of BR-based processes and applications can be found in Hampp. Most applications utilize the photochromic properties of BR. Two of these processes are discussed in this chapter the two-dimensional holographic memory and the three-dimensional associative memory. However, other bio-molecular photonic applications are currently being examined. BR is also being used in the process or formation of desahnation of seawater, conversion of sunlight into electricity, artificial retinae, motion detection, optical filtering, neural networking, radiation detection, and biosensor applications. The universal application of BR in many different biomaterials is testament to the flexibility and utility of the protein. 

---