Molecular associative memory

An example of a kinetic system with address-bearing molecules and directed interactions is investigated. We show that by introducing exchange between Ising spins via the address-bearing messengers, the Hopfield model of associative memory can be made local.     

Error tolerant associative memory

We present a new approach to enlarging the basin of attraction of associative memory, including auto-associative memory and temporal associative memory. The memory trained by means of this method can tolerate and recover from seriously noisy patterns. Simulations show that this approach will greatly reduce the number of limit cycles.     

A cognitive and associative memory

By introducing a physiological constraint in the auto-correlation matrix memory, the system is found to acquire an ability in cognition i.e. the ability to identify and input pattern by its proximity to any one of the stored memories. The physiological constraint here is that the attribute of a given synapse (i.e. excitatory or inhibitory) is uniquely determined by the neuron it belongs. Thus the synaptic coupling is generally not symmetric. Analytical and numerical analyses revealed that the present model retrieves a memory if an input pattern is close to the pattern of the stored memories; if not, it gives a clear response by going into a special mode where almost all neurons are in the same state in each time step. This uniform mode may be stationary or periodic, depending on whether or not the number of the excitatory neurons exceeds the number of inhibitory neurons.     

Internal representations for associative memory

We describe a class of feed forward neural network models for associative content addressable memory (ACAM) which utilize sparse internal representations for stored data. In addition to the input and output layers, our networks incorporate an intermediate processing layer which serves to label each stored memory and to perform error correction and association. We study two classes of internal label representations: the unary representation and various sparse, distributed representations. Finally, we consider storage of sparse data and sparsification of data. These models are found to have advantages in terms of storage capacity, hardware efficiency, and recall reliability when compared to the Hopfield model, and to possess analogies to both biological neural networks and standard digital computer memories.     

Visual homing using an associative memory

Homing is the process by which an autonomous system guides itself to a particular location on the basis of sensory input. In this paper, a method of visual homing using an associative memory based on a simple pattern classifier is described. Homing is accomplished without the use of an explicit world model by utilizing direct associations between learned visual patterns and system motor commands. The method is analyzed in terms of a pattern space and conditions obtained that allow the system performance to be predicted on the basis of statistical measurements on the environment. Results of experiments utilizing the method to guide a robot-mounted camera in a three-dimensional environment are presented.     

Improving recall from an associative memory

The associative net as a model of biological associative memory is investigated. Calculating the output pattern retrieved from a partially connected associative net presented with noisy input cues involves several computations. This is complicated by variations in the dendritic sums of the output units due to errors in the cue and differences in input activity and unit usage. The possible implementation of these computations by biological neural machinery is unclear. We demonstrate that a relatively simple transformation can reduce variation in the dendritic sums. This leads to a winners-take-all type of strategy that produces increased recall performance which is equivalent to the more complicated optimal strategy proposed by others. We describe in detail the possible biological implications of our strategies, the novel feature of which ascribes a role to the NMDA and non-NMDA channels found in hippocampal pyramidal cells. Received: 13 April 1994 / Accepted: 25 October 1994     

On the storage capacity of an associative memory with randomly distributed storage elements

In an associative memory with randomly distributed storage elements at least 0.05 bit per storage element can be stored.     

Stability and attractivity in associative memory networks

We focus on stable and attractive states in a network having two-state neuron-like elements. We calculate the connection matrix which guarantees the stability and the strongest attractivity of p memorized patterns. We present an analytical evaluation of the patterns' attractivity. These results are illustrated by some computer simulations.     

A holographic model for associative memory chains

Holographic brain models are well suited to describe specific brain functions. Central nervous systems and holographic systems both show parallel information processing and non-localized storage in common. To process information both systems use correlation functions suggesting to develop cybernetical brain models in terms of holography. Associative holographic storage is done with two simultaneously existing patterns. They may reconstruct each other mutually. Time-sequentially existing patterns are connected to associative chains, if every two succeeding patterns do exist within a common period of time in order to be stored in pairs. Read out (recall) of associative chains—reconstructing coupled patterns which didn't exist simultaneously—requires advanced holographic techniques. Three different methods are described and tested experimentally. The underlying principles are feedback mechanisms, nonlinearities of the storage material and tridimensional architecture of the voluminous recording medium. Those principles evidently occur in neural storage systems supporting analogous information processing in neural- and holographic systems.     

Pseudo-relaxation learning algorithm for complex-valued associative memory

HAM (Hopfield Associative Memory) and BAM (Bidirectinal Associative Memory) are representative associative memories by neural networks. The storage capacity by the Hebb rule, which is often used, is extremely low. In order to improve it, some learning methods, for example, pseudo-inverse matrix learning and gradient descent learning, have been introduced. Oh introduced pseudo-relaxation learning algorithm to HAM and BAM. In order to accelerate it, Hattori proposed quick learning. Noest proposed CAM (Complex-valued Associative Memory), which is complex-valued HAM. The storage capacity of CAM by the Hebb rule is also extremely low. Pseudo-inverse matrix learning and gradient descent learning have already been generalized to CAM. In this paper, we apply pseudo-relaxation learning algorithm to CAM in order to improve the capacity.     

Crossmodal associative memory representations in rodent orbitofrontal cortex

Firing patterns of neurons in the orbitofrontal cortex (OF) were analyzed in rats trained to perform a task that encouraged incidental associations between distinct odors and the places where their occurrence was detected. Many of the neurons fired differentially when the animals were at a particular location or sampled particular odors. Furthermore, a substantial fraction of the cells exhibited odor-specific firing patterns prior to odor presentation, when the animal arrived at a location associated with that odor. These findings suggest that neurons in the OF encode cross-modal associations between odors and locations within long-term memory.     

Synaptic mechanisms of associative memory in the amygdala

Do associative learning and synaptic long-term potentiation (LTP) depend on the same cellular mechanisms? Recent work in the amygdala reveals that LTP and Pavlovian fear conditioning induce similar changes in postsynaptic AMPA-type glutamate receptors and that occluding these changes by viral-mediated overexpression of a dominant-negative GluR1 construct attenuates both LTP and fear memory in rats. Novel forms of presynaptic plasticity in the lateral nucleus may also contribute to fear memory formation, bolstering the connection between synaptic plasticity mechanisms and associative learning and memory.     

A hierarchical neural network model for associative memory

A hierarchical neural network model with feedback interconnections, which has the function of associative memory and the ability to recognize patterns, is proposed. The model consists of a hierarchical multi-layered network to which efferent connections are added, so as to make positive feedback loops in pairs with afferent connections. The cell-layer at the initial stage of the network is the input layer which receives the stimulus input and at the same time works as an output layer for associative recall. The deepest layer is the output layer for pattern-recognition. Pattern-recognition is performed hierarchically by integrating information by converging afferent paths in the network. For the purpose of associative recall, the integrated information is again distributed to lower-order cells by diverging efferent paths. These two operations progress simultaneously in the network. If a fragment of a training pattern is presented to the network which has completed its self-organization, the entire pattern will gradually be recalled in the initial layer. If a stimulus consisting of a number of training patterns superposed is presented, one pattern gradually becomes predominant in the recalled output after competition between the patterns, and the others disappear. At about the same time when the recalled pattern reaches a steady state in he initial layer, in the deepest layer of the network, a response is elicited from the cell corresponding to the category of the finally-recalled pattern. Once a steady state has been reached, the response of the network is automatically extinguished by inhibitory signals from a steadiness-detecting cell. If the same stimulus is still presented after inhibition, a response for another pattern, formerly suppressed, will now appear, because the cells of the network have adaptation characteristics which makes the same response unlikely to recur. Since inhibition occurs repeatedly, the superposed input patterns are recalled one by one in turn.     

Recollection ability of three-dimensional correlation matrix associative memory

This paper presents the approximate expression of the recollection probability of a three-dimensional correlation matrix autoassociative memory, in which each memorized pattern consists of binary (+1 or-1) elements, and discusses the recollection ability in comparison with that of the conventional two-dimensional correlation matrix associative memory. An associative memory using the correlation properties between memorized patterns desires the condition that the memorized patterns are mutually orthogonal or approximately orthogonal. The three-dimensional correlation matrix associative memory that inscribes the third order correlation of a memorized pattern heightens the recollection ability even though the above condition is not satisfied.     

Acetylcholine and associative memory in the piriform cortex

The significance of cholinergic modulation for associative memory performance in the piriform cortex was examined in a study combining cellular neurophysiology in brain slices with realistic biophysical network simulations. Three different physiological effects of acetylcholine were identified at the single-cell level: suppression of neuronal adaptation, suppression of synaptic transmission in the intrinsic fibers layer, and activity-dependent increase in synaptic strength. Biophysical simulations show how these three effects are joined together to enhance learning and recall performance of the cortical network. Furthermore, our data suggest that activity-dependent synaptic decay during learning is a crucial factor in determining learning capability of the cortical network. Accordingly, it is predicted that acetylcholine should also enhance long-term depression in the piriform cortex.     

A model of associative memory in the brain

A model of associative memory for time varying spatial patterns is proposed and simulated on a digital computer. This is a network composed of many neuron-like elements, and shows an ability for associative memory similar to that of the brain.Suppose a number of sequences of spatial patterns are presented to this network, for example, 12345, ABC, and so on. Then, these patterns are memorized in the network. After that, if any part of one of these sequences, say 23, is presented to the circuit, the rest of the sequence, 45, is recalled following to it. It resembles to such a situation — if we hear a part of a melody which we have memorized in the past, the rest of the melody is recalled even after it is stopped half-way. Although the recalled patterns are not always 100% correct, they are not completely destroyed even if the presented patterns are imperfect.     

Visual associative memory and the orientation-contingent color after-effect

The traditional explanation of the McCollough effect (ME) by selective adaptation of single detectors selective to color and orientation suffers from a number of inconsistencies: 1) the ME lasts much longer (from several days up to 3 months) than the ordinary adaptation, the decay of the effect being completely arrested by night sleep or occluding the eye for a long time; 2) the strength of the ME practically does not depend on the intensity of adapting light; and 3) a set of related pattern-contingent after-effects discovered later required for such an explanation new detectors, specific for other patterns. These properties can be explained, however, in the framework of associative memory and novelty filters. A computational model has been developed, which consists of 1) an input layer of two (left and right eyes) square matrices with two analog receptors (red and green) in each pixel, 2) an isomorphic associative neural layer, each analog neuron being synaptically connected with all receptors of both eyes, and 3) an output layer (novelty filter). The modification of synaptic efficacies conforms to the Hebb learning rule. The function of the model was examined by simulation. After a few presentations of colored gratings, the model displays the ME that is slowly destroyed by subsequent presentations of random pictures. With a sufficiently large receptor matrix, the effect lasts a thousand times longer than the period of adaptation. Continuous darkness does not change the strength of the effect. Like in real ME, the model does not display interocular transfer. The model can account for different pattern-contingent color after-effects without assuming any predetermined specific detectors. Such detectors are constructed in the course of adaptation to specific stimuli (gratings).     

Associative search network: A reinforcement learning associative memory

An associative memory system is presented which does not require a teacher to provide the desired associations. For each input key it conducts a search for the output pattern which optimizes an external payoff or reinforcement signal. The associative search network (ASN) combines pattern recognition and function optimization capabilities in a simple and effective way. We define the associative search problem, discuss conditions under which the associative search network is capable of solving it, and present results from computer simulations. The synthesis of sensory-motor control surfaces is discussed as an example of the associative search problem.     

Dietary choline augments associative memory function in Limax maximus

Previous clinical and experimental work has shown that increased dietary intake of choline elevates blood choline and brain acetylcholine levels. This change in neuronal acetylcholine concentration may augment learning and memory functions. We tested this prediction using the mollusc Limax maximus, an animal which can be readily conditioned to avoid food odors. In our experiments, initial learning of a food avoidance task was not augmented by the high choline diet. However, the duration of memory retention was prolonged. In previous studies, we have shown that intake of the choline enriched diet significantly increases blood choline and amplifies transmission at an identified cholinergic synapse in Limax. Together, these results support the involvement of cholinergic synapses in the memory retention mechanism.     

An algebraic model of an associative noise-like coding memory

A mathematical model of an associative memory is presented, sharing with the optical holography memory systems the properties which establish an analogy with biological memory. This memory system-developed from Gabor's model of memoryis based on a noise-like coding of the information by which it realizes a distributed, damage-tolerant, equipotential storage through simultaneous state changes of discrete substratum elements. Each two associated items being stored are coded by each other by means of two noise-like patterns obtained from them through a randomizing preprocessing. The algebraic braic transformations operating the information storage and retrieval are matrix-vector products involving Toeplitz type matrices. Several noise-like coded memory traces are superimposed on a common substratum without crosstalk interference; moreover, extraneous noise added to these memory traces does not injure the stored information. The main performances shown by this memory model are: i) the selective, complete recovering of stored information from incomplete keys, both mixed with extraneous information and translated from the position learnt; ii) a dynamic recollection where the information just recovered acts as a new key for a sequential retrieval process; iii) context-dependent responses. The hypothesis that the information is stored in the nervous system through a noise-like coding is suggested. The model has been simulated on a digital computer using bidimensional images.