Dynamics control of semantic processes in a hierarchical associative memory

 A neural mechanism for control of dynamics and function of associative processes in a hierarchical memory system is demonstrated. For the representation and processing of abstract knowledge, the semantic declarative memory system of the human brain is considered. The dynamics control mechanism is based on the influence of neuronal adaptation on the complexity of neural network dynamics. Different dynamical modes correspond to different levels of the ultrametric structure of the hierarchical memory being invoked during an associative process. The mechanism is deterministic but may also underlie free associative thought processes. The formulation of an abstract neural network model of hierarchical associative memory utilizes a recent approach to incorporate neuronal adaptation. It includes a generalized neuronal activation function recently derived by a Hodgkin-Huxley-type model. It is shown that the extent to which a hierarchically organized memory structure is searched is controlled by the neuronal adaptability, i.e. the strength of coupling between neuronal activity and excitability. In the brain, the concentration of various neuromodulators in turn can regulate the adaptability. An autonomously controlled sequence of bifurcations, from an initial exploratory to a final retrieval phase, of an associative process is shown to result from an activity-dependent release of neuromodulators. The dynamics control mechanism may be important in the context of various disorders of the brain and may also extend the range of applications of artificial neural networks. Received: 19 April 1995/Accepted in revised form: 8 August 1995     

A hierarchical neural network model for associative memory

A hierarchical neural network model with feedback interconnections, which has the function of associative memory and the ability to recognize patterns, is proposed. The model consists of a hierarchical multi-layered network to which efferent connections are added, so as to make positive feedback loops in pairs with afferent connections. The cell-layer at the initial stage of the network is the input layer which receives the stimulus input and at the same time works as an output layer for associative recall. The deepest layer is the output layer for pattern-recognition. Pattern-recognition is performed hierarchically by integrating information by converging afferent paths in the network. For the purpose of associative recall, the integrated information is again distributed to lower-order cells by diverging efferent paths. These two operations progress simultaneously in the network. If a fragment of a training pattern is presented to the network which has completed its self-organization, the entire pattern will gradually be recalled in the initial layer. If a stimulus consisting of a number of training patterns superposed is presented, one pattern gradually becomes predominant in the recalled output after competition between the patterns, and the others disappear. At about the same time when the recalled pattern reaches a steady state in he initial layer, in the deepest layer of the network, a response is elicited from the cell corresponding to the category of the finally-recalled pattern. Once a steady state has been reached, the response of the network is automatically extinguished by inhibitory signals from a steadiness-detecting cell. If the same stimulus is still presented after inhibition, a response for another pattern, formerly suppressed, will now appear, because the cells of the network have adaptation characteristics which makes the same response unlikely to recur. Since inhibition occurs repeatedly, the superposed input patterns are recalled one by one in turn.     

Pattern recognition and associative memory as dynamical processes in a synergetic system

We first present numerical results for the decomposition procedure for complex scenes described in Part I of these papers. Part II then mainly deals with a formalism that allows a formulation of our approach to pattern recognition and associative memory that is simultaneously invariant against translation, rotation, and scaling, Part II thus contains an explicit elaboration of ideas of Part I.     

Pattern recognition and associative memory as dynamical processes in a synergetic system

We consider a model for associative memory and pattern recognition which was devised by Haken (1987b). This model treats the activity of the neurons as continuous variables and exploits an analogy with pattern formation in synergetic systems. The capability of such a system to act as associative memory is demonstrated by the reconstruction of faces which are partially offered to the system, and which are restored by the corresponding dynamical process. We demonstrate how this model can be cast into a form which is translation invariant and how partially hidden faces in scenes can be recognized by means of the control of attention parameters of specific patterns.     

Recurrent loops: Incorporating prediction error and semantic/episodic theories into Drosophila associative memory models

In 2003, Martin Heisenberg et al. presented a model of how associative memories could be encoded and stored in the insect brain. This model was extremely influential in the Drosophila memory field, but did not incorporate several important mammalian concepts, including ideas of separate episodic and semantic types of memory and prediction error hypotheses. In addition, at that time, the concept of memory traces recurrently entering and exiting the mushroom bodies, brain areas where associative memories are formed and stored, was unknown. In this review, I present a simple updated model incorporating these ideas, which may be useful for future studies.     

Cluster associative memory formation in a three-layer network of phase oscillators

A three-layer network model of oscillatory associative memory is proposed. The network is capable of storing binary images, which can be retrieved upon presenting an appropriate stimulus. Binary images are encoded in the form of the spatial distribution of oscillatory phase clusters in-phase and anti-phase relative to a reference periodic signal. The information is loaded into the network using a set of interlayer connection weights. A condition for error-free pattern retrieval is formulated, delimiting the maximal number of patterns to be stored in the memory (storage capacity). It is shown that the capacity can be significantly increased by generating an optimal alphabet (basis pattern set). The number of stored patterns can reach values of the network size (the number of oscillators in each layer), which is significantly higher than the capacity of conventional oscillatory memory models. The dynamical and information characteristics of the retrieval process based on the optimal alphabet, including the size of “attraction basins“ and the input pattern distortion admissible for error-free retrieval, are investigated.     

Stability and attractivity in associative memory networks

We focus on stable and attractive states in a network having two-state neuron-like elements. We calculate the connection matrix which guarantees the stability and the strongest attractivity of p memorized patterns. We present an analytical evaluation of the patterns' attractivity. These results are illustrated by some computer simulations.     

Mixed state on a sparsely encoded associative memory model

 We analyzed symmetric mixed states corresponding to the so-called concept formation on a sparsely encoded associative memory model with 0–1 neurons. Three types of mixed states – OR, AND, and a majority decision – are described as typical examples. Each element of the OR mixed state is composed of corresponding memory pattern elements by means of the OR operation. The other two types are similarly defined. By analyzing their equilibrium properties through self-consistent signal-to-noise analysis and computer simulation, we found that the storage capacity of the OR mixed state diverges in a sparse limit, but that the other states do not diverge. In addition, we found that the optimal threshold values, which maximize the storage capacity for the memory pattern and the OR mixed state, coincide with each other in the spare limit. We conclude that the OR mixed state is a reasonable representative of mixed state in the sparse limit. Received: 10 November 1999 / Accepted in revised form: 5 April 2001     

Synaptic mechanisms of associative memory in the amygdala

Do associative learning and synaptic long-term potentiation (LTP) depend on the same cellular mechanisms? Recent work in the amygdala reveals that LTP and Pavlovian fear conditioning induce similar changes in postsynaptic AMPA-type glutamate receptors and that occluding these changes by viral-mediated overexpression of a dominant-negative GluR1 construct attenuates both LTP and fear memory in rats. Novel forms of presynaptic plasticity in the lateral nucleus may also contribute to fear memory formation, bolstering the connection between synaptic plasticity mechanisms and associative learning and memory.     

Trace feature map: a model of episodic associative memory

An approach to episodic associative memory is presented, which has several desirable properties as a human memory model. The design is based on topological feature map representation of data. An ordinary feature map is a classifier, mapping an input vector onto a topologically meaningful location on the map. A trace feature map, in addition, creates a memory trace on that location. The traces can be stored episodically in a single presentation, and retrieved with a partial cue. Nearby traces overlap, which results in plausible memory interference behavior. Performance degrades gracefully as the memory is overloaded. More recent traces are easier to recall as are traces that are unique in the memory.This research was supported in part by an ITA Foundation grant and by fellowships from the Academy of Finland, the Emil Aaltonen Foundation and the Foundation for the Advancement of Technology (Finland) when the author was at UCLA     

A model of associative memory in the brain

A model of associative memory for time varying spatial patterns is proposed and simulated on a digital computer. This is a network composed of many neuron-like elements, and shows an ability for associative memory similar to that of the brain.Suppose a number of sequences of spatial patterns are presented to this network, for example, 12345, ABC, and so on. Then, these patterns are memorized in the network. After that, if any part of one of these sequences, say 23, is presented to the circuit, the rest of the sequence, 45, is recalled following to it. It resembles to such a situation — if we hear a part of a melody which we have memorized in the past, the rest of the melody is recalled even after it is stopped half-way. Although the recalled patterns are not always 100% correct, they are not completely destroyed even if the presented patterns are imperfect.     

Influence of oxytocin on human memory processes: Validation by a control study

An earlier report (1) of an adverse effect of high doses of oxytocin on human memory included results of studies on women receiving oxytocin as part of the treatment to induce 2nd trimester therapeutic abortion. These women served as their own controls. We have now been able to study a group of women who have been treated in all ways like the original group, with the exception that they did not receive oxytocin. The results from this external control corroborate the finding that oxytocin affected memory.     

Frontally mediated control processes contribute to source memory retrieval

Remembering is a cognitively demanding task that requires the strategic selection of information from memory. In this issue of Neuron, Dobbins et al. present functional MRI (fMRI) data that shed insight into the specific, dissociated contributions of frontal regions to remembering.     

Crossmodal associative memory representations in rodent orbitofrontal cortex

Firing patterns of neurons in the orbitofrontal cortex (OF) were analyzed in rats trained to perform a task that encouraged incidental associations between distinct odors and the places where their occurrence was detected. Many of the neurons fired differentially when the animals were at a particular location or sampled particular odors. Furthermore, a substantial fraction of the cells exhibited odor-specific firing patterns prior to odor presentation, when the animal arrived at a location associated with that odor. These findings suggest that neurons in the OF encode cross-modal associations between odors and locations within long-term memory.     

Efficient and robust associative memory from a generalized Bloom filter

We develop a variant of a Bloom filter that is robust to hardware failure and show how it can be used as an efficient associative memory. We define a measure of the information recall and show that our new associative memory is able to recall more than twice as much information as a Hopfield network. The extra efficiency of our associative memory is all the more remarkable as it uses only bits while the Hopfield network uses integers.     

Acetylcholine and associative memory in the piriform cortex

The significance of cholinergic modulation for associative memory performance in the piriform cortex was examined in a study combining cellular neurophysiology in brain slices with realistic biophysical network simulations. Three different physiological effects of acetylcholine were identified at the single-cell level: suppression of neuronal adaptation, suppression of synaptic transmission in the intrinsic fibers layer, and activity-dependent increase in synaptic strength. Biophysical simulations show how these three effects are joined together to enhance learning and recall performance of the cortical network. Furthermore, our data suggest that activity-dependent synaptic decay during learning is a crucial factor in determining learning capability of the cortical network. Accordingly, it is predicted that acetylcholine should also enhance long-term depression in the piriform cortex.     

Models of distributed associative memory networks in the brain

Summary Although experimental evidence for distributed cell assemblies is growing, theories of cell assemblies are still marginalized in theoretical neuroscience. We argue that this has to do with shortcomings of the currently best understood assembly theories, the ones based on formal associative memory models. These only insufficiently reflect anatomical and physiological properties of nervous tissue, and their functionality is too restricted to provide a framework for cognitive modeling. We describe cell assembly models that integrate more neurobiological constraints and review results from simulations of a simple nonlocal associative network formed by a reciprocal topographic projection. Impacts of nonlocal associative projections in the brain are discussed with respect to the functionality they can explain.