A holographic model for associative memory chains

Holographic brain models are well suited to describe specific brain functions. Central nervous systems and holographic systems both show parallel information processing and non-localized storage in common. To process information both systems use correlation functions suggesting to develop cybernetical brain models in terms of holography. Associative holographic storage is done with two simultaneously existing patterns. They may reconstruct each other mutually. Time-sequentially existing patterns are connected to associative chains, if every two succeeding patterns do exist within a common period of time in order to be stored in pairs. Read out (recall) of associative chains—reconstructing coupled patterns which didn't exist simultaneously—requires advanced holographic techniques. Three different methods are described and tested experimentally. The underlying principles are feedback mechanisms, nonlinearities of the storage material and tridimensional architecture of the voluminous recording medium. Those principles evidently occur in neural storage systems supporting analogous information processing in neural- and holographic systems.     

A hierarchical neural network model for associative memory

A hierarchical neural network model with feedback interconnections, which has the function of associative memory and the ability to recognize patterns, is proposed. The model consists of a hierarchical multi-layered network to which efferent connections are added, so as to make positive feedback loops in pairs with afferent connections. The cell-layer at the initial stage of the network is the input layer which receives the stimulus input and at the same time works as an output layer for associative recall. The deepest layer is the output layer for pattern-recognition. Pattern-recognition is performed hierarchically by integrating information by converging afferent paths in the network. For the purpose of associative recall, the integrated information is again distributed to lower-order cells by diverging efferent paths. These two operations progress simultaneously in the network. If a fragment of a training pattern is presented to the network which has completed its self-organization, the entire pattern will gradually be recalled in the initial layer. If a stimulus consisting of a number of training patterns superposed is presented, one pattern gradually becomes predominant in the recalled output after competition between the patterns, and the others disappear. At about the same time when the recalled pattern reaches a steady state in he initial layer, in the deepest layer of the network, a response is elicited from the cell corresponding to the category of the finally-recalled pattern. Once a steady state has been reached, the response of the network is automatically extinguished by inhibitory signals from a steadiness-detecting cell. If the same stimulus is still presented after inhibition, a response for another pattern, formerly suppressed, will now appear, because the cells of the network have adaptation characteristics which makes the same response unlikely to recur. Since inhibition occurs repeatedly, the superposed input patterns are recalled one by one in turn.     

Models of distributed associative memory networks in the brain

Summary Although experimental evidence for distributed cell assemblies is growing, theories of cell assemblies are still marginalized in theoretical neuroscience. We argue that this has to do with shortcomings of the currently best understood assembly theories, the ones based on formal associative memory models. These only insufficiently reflect anatomical and physiological properties of nervous tissue, and their functionality is too restricted to provide a framework for cognitive modeling. We describe cell assembly models that integrate more neurobiological constraints and review results from simulations of a simple nonlocal associative network formed by a reciprocal topographic projection. Impacts of nonlocal associative projections in the brain are discussed with respect to the functionality they can explain.     

On the storage capacity of an associative memory with randomly distributed storage elements

In an associative memory with randomly distributed storage elements at least 0.05 bit per storage element can be stored.     

A cognitive and associative memory

By introducing a physiological constraint in the auto-correlation matrix memory, the system is found to acquire an ability in cognition i.e. the ability to identify and input pattern by its proximity to any one of the stored memories. The physiological constraint here is that the attribute of a given synapse (i.e. excitatory or inhibitory) is uniquely determined by the neuron it belongs. Thus the synaptic coupling is generally not symmetric. Analytical and numerical analyses revealed that the present model retrieves a memory if an input pattern is close to the pattern of the stored memories; if not, it gives a clear response by going into a special mode where almost all neurons are in the same state in each time step. This uniform mode may be stationary or periodic, depending on whether or not the number of the excitatory neurons exceeds the number of inhibitory neurons.     

An algebraic model of an associative noise-like coding memory

A mathematical model of an associative memory is presented, sharing with the optical holography memory systems the properties which establish an analogy with biological memory. This memory system-developed from Gabor's model of memoryis based on a noise-like coding of the information by which it realizes a distributed, damage-tolerant, equipotential storage through simultaneous state changes of discrete substratum elements. Each two associated items being stored are coded by each other by means of two noise-like patterns obtained from them through a randomizing preprocessing. The algebraic braic transformations operating the information storage and retrieval are matrix-vector products involving Toeplitz type matrices. Several noise-like coded memory traces are superimposed on a common substratum without crosstalk interference; moreover, extraneous noise added to these memory traces does not injure the stored information. The main performances shown by this memory model are: i) the selective, complete recovering of stored information from incomplete keys, both mixed with extraneous information and translated from the position learnt; ii) a dynamic recollection where the information just recovered acts as a new key for a sequential retrieval process; iii) context-dependent responses. The hypothesis that the information is stored in the nervous system through a noise-like coding is suggested. The model has been simulated on a digital computer using bidimensional images.     

Mixed state on a sparsely encoded associative memory model

 We analyzed symmetric mixed states corresponding to the so-called concept formation on a sparsely encoded associative memory model with 0–1 neurons. Three types of mixed states – OR, AND, and a majority decision – are described as typical examples. Each element of the OR mixed state is composed of corresponding memory pattern elements by means of the OR operation. The other two types are similarly defined. By analyzing their equilibrium properties through self-consistent signal-to-noise analysis and computer simulation, we found that the storage capacity of the OR mixed state diverges in a sparse limit, but that the other states do not diverge. In addition, we found that the optimal threshold values, which maximize the storage capacity for the memory pattern and the OR mixed state, coincide with each other in the spare limit. We conclude that the OR mixed state is a reasonable representative of mixed state in the sparse limit. Received: 10 November 1999 / Accepted in revised form: 5 April 2001     

Trace feature map: a model of episodic associative memory

An approach to episodic associative memory is presented, which has several desirable properties as a human memory model. The design is based on topological feature map representation of data. An ordinary feature map is a classifier, mapping an input vector onto a topologically meaningful location on the map. A trace feature map, in addition, creates a memory trace on that location. The traces can be stored episodically in a single presentation, and retrieved with a partial cue. Nearby traces overlap, which results in plausible memory interference behavior. Performance degrades gracefully as the memory is overloaded. More recent traces are easier to recall as are traces that are unique in the memory.This research was supported in part by an ITA Foundation grant and by fellowships from the Academy of Finland, the Emil Aaltonen Foundation and the Foundation for the Advancement of Technology (Finland) when the author was at UCLA     

Associative search network: A reinforcement learning associative memory

An associative memory system is presented which does not require a teacher to provide the desired associations. For each input key it conducts a search for the output pattern which optimizes an external payoff or reinforcement signal. The associative search network (ASN) combines pattern recognition and function optimization capabilities in a simple and effective way. We define the associative search problem, discuss conditions under which the associative search network is capable of solving it, and present results from computer simulations. The synthesis of sensory-motor control surfaces is discussed as an example of the associative search problem.     

Topological feature maps with self-organized lateral connections: a population-coded,one-layer model of associative memory

Guided by the neurobiological principles of self-organization and population coding, we develop a simple, neural, one-layer model for auto-association. Its core is a feature map endowed with self-organized lateral connections. Input patterns are coded by small spots of active neurons. The time evolution of neural activity then realizes an auto-association process by a recurrent attractor dynamics. Population coding is preserved due to a balance of diffusive spreading of activity and competitive refocusing. Because of its simplicity, the model allows a thorough qualitative and quantitative understanding. We show that the network is capable of performing a cluster analysis and hierarchical classification of data and, thus, qualifies as a tool for unsupervised statistical data analysis.     

Coexistence of memory patterns and mixed states in a sparsely encoded associative memory model storing ultrametric patterns

Analyzing the coexistence of memory patterns and mixed states gives important information for constructing a model for the face responsive neurons of the monkey inferior-temporal cortex. We analyzed whether the memory patterns coexist with mixed states when the sparse coding scheme is used for the associative memory model storing ultrametric patterns. For memory patterns and mixed states to coexist, there must be sufficient capacity for storing them and their threshold values must be the same. We determined that the storage capacities for all mixed states composed of correlated memory patterns diverge as 1/|flogf| (where f is the firing rate) even when the correlation of the memory patterns is infinitesimally small. We also determined that the memory patterns and the mixed states can become the equilibrium state of the model in the same threshold value. These results mean that they can coexist in this model. These findings should contribute to research on face responsive neurons in the monkey inferior-temporal cortex.     

A new neural network architecture with associative memory, pruning and order-sensitive learning

A new paradigm of neural network architecture is proposed that works as associative memory along with capabilities of pruning and order-sensitive learning. The network has a composite structure wherein each node of the network is a Hopfield network by itself. The Hopfield network employs an order-sensitive learning technique and converges to user-specified stable states without having any spurious states. This is based on geometrical structure of the network and of the energy function. The network is so designed that it allows pruning in binary order as it progressively carries out associative memory retrieval. The capacity of the network is 2n, where n is the number of basic nodes in the network. The capabilities of the network are demonstrated by experimenting on three different application areas, namely a Library Database, a Protein Structure Database and Natural Language Understanding.     

A complex-valued associative memory for storing patterns as oscillatory states

A neuron model in which the neuron state is described by a complex number is proposed. A network of these neurons, which can be used as an associative memory, operates in two distinct modes: (i) fixed point mode and (ii) oscillatory mode. Mode selection can be done by varying a continuous mode parameter, , between and . At one extreme value of (), the network has conservative dynamics, and at the other (), the dynamics are dissipative and governed by a Lyapunov function. Patterns can be stored and retrieved at any value of by, (i) a one-step outer product rule or (ii) adaptive Hebbian learning. In the fixed point mode patterns are stored as fixed points, whereas in the oscillatory mode they are encoded as phase relations among individual oscillations. By virtue of an instability in the oscillatory mode, the retrieval pattern is stable over a finite interval, the stability interval, and the pattern gradually deteriorates with time beyond this interval. However, at certain values of sparsely distributed over -space the instability disappears. The neurophysiological significance of the instability is briefly discussed. The possibility of physically interpreting dissipativity and conservativity is explored by noting that while conservativity leads to energy savings, dissipativity leads to stability and reliable retrieval. Received: 4 December 1995 / Accepted in revised form: 18 June 1996     

An oscillatory neural network model of sparse distributed memory and novelty detection

A model of sparse distributed memory is developed that is based on phase relations between the incoming signals and an oscillatory mechanism for information processing. This includes phase-frequency encoding of input information, natural frequency adaptation among the network oscillators for storage of input signals, and a resonance amplification mechanism that responds to familiar stimuli. Simulations of this model show different types of dynamics in response to new and familiar stimuli. The application of the model to hippocampal working memory is discussed.     

Neuronal model with distributed delay: analysis and simulation study for gamma distribution memory kernel

A single neuronal model incorporating distributed delay (memory)is proposed. The stochastic model has been formulated as a Stochastic Integro-Differential Equation (SIDE) which results in the underlying process being non-Markovian. A detailed analysis of the model when the distributed delay kernel has exponential form (weak delay) has been carried out. The selection of exponential kernel has enabled the transformation of the non-Markovian model to a Markovian model in an extended state space. For the study of First Passage Time (FPT) with exponential delay kernel, the model has been transformed to a system of coupled Stochastic Differential Equations (SDEs) in two-dimensional state space. Simulation studies of the SDEs provide insight into the effect of weak delay kernel on the Inter-Spike Interval(ISI) distribution. A measure based on Jensen-Shannon divergence is proposed which can be used to make a choice between two competing models viz. distributed delay model vis-á-vis LIF model. An interesting feature of the model is that the behavior of (CV(t))((ISI)) (Coefficient of Variation) of the ISI distribution with respect to memory kernel time constant parameter η reveals that neuron can switch from a bursting state to non-bursting state as the noise intensity parameter changes. The membrane potential exhibits decaying auto-correlation structure with or without damped oscillatory behavior depending on the choice of parameters. This behavior is in agreement with empirically observed pattern of spike count in a fixed time window. The power spectral density derived from the auto-correlation function is found to exhibit single and double peaks. The model is also examined for the case of strong delay with memory kernel having the form of Gamma distribution. In contrast to fast decay of damped oscillations of the ISI distribution for the model with weak delay kernel, the decay of damped oscillations is found to be slower for the model with strong delay kernel.     

A model of localized iconic memory, replicable and associative, using time sharing and 3 neuronal communication modes

A model is proposed for the functioning of an iconic memory involving several layers of neurones. A small group of neurones in one layer project their terminations over the terminations of a single neurone of the superior layer. According to the communication mode (emission or reception), a neurone in one layer can memorize the state of the terminations of a neurone of the superior layer, or impose on the latter the state of its own terminations. In the comparison mode, an emitting neurone compares its state to another emitting neurone and, in case of sufficient similarity, switches to the reception mode (associative recall). The first layer, corresponding to short-term memory, communicates with the cells involved in the representation of the perceived image. This model makes possible the establishment of a correspondence between a percept and a neurone, the replication of memorized configurations, the restructuration of memory and, starting with a percept or a memorized item, the integral associative recall of all similar memorized items.     

A model of associative learning in the mushroom body

The mushroom body is a prominent invertebrate neuropil strongly associated with learning and memory. We built a high-level computational model of this structure using simplified but realistic models of neurons and synapses, and developed a learning rule based on activity dependent pre-synaptic facilitation. We show that our model, which is consistent with mushroom body Drosophila data and incorporates Aplysia learning, is able to both acquire and later recall CS-US associations. We demonstrate that a highly divergent input connectivity to the mushroom body and strong periodic inhibition both serve to improve overall learning performance. We also examine the problem of how synaptic conductance, driven by successive training events, obtains a value appropriate for the stimulus being learnt. We employ two feedback mechanisms: one stabilises strength at an initial level appropriate for an association; another prevents strength increase for established associations.     

Interfering with theories of sleep and memory: sleep, declarative memory, and associative interference

Mounting behavioral evidence in humans supports the claim that sleep leads to improvements in recently acquired, nondeclarative memories. Examples include motor-sequence learning; visual-discrimination learning; and perceptual learning of a synthetic language. In contrast, there are limited human data supporting a benefit of sleep for declarative (hippocampus-mediated) memory in humans (for review, see). This is particularly surprising given that animal models (e.g.,) and neuroimaging studies (e.g.,) predict that sleep facilitates hippocampus-based memory consolidation. We hypothesized that we could unmask the benefits of sleep by challenging the declarative memory system with competing information (interference). This is the first study to demonstrate that sleep protects declarative memories from subsequent associative interference, and it has important implications for understanding the neurobiology of memory consolidation.     

Synaptic mechanisms of associative memory in the amygdala

Do associative learning and synaptic long-term potentiation (LTP) depend on the same cellular mechanisms? Recent work in the amygdala reveals that LTP and Pavlovian fear conditioning induce similar changes in postsynaptic AMPA-type glutamate receptors and that occluding these changes by viral-mediated overexpression of a dominant-negative GluR1 construct attenuates both LTP and fear memory in rats. Novel forms of presynaptic plasticity in the lateral nucleus may also contribute to fear memory formation, bolstering the connection between synaptic plasticity mechanisms and associative learning and memory.     

Full-system simulation of distributed memory multicomputers

In this paper we discuss environments for the full-system simulation of multicomputers. These environments are composed of a large collection of modules that simulate the compute nodes and the network, plus additional linking elements that perform communication and synchronization. We present our own environment, in which we integrate Simics with INSEE. We reuse as many Simics modules as possible to reduce the effort of hardware modeling, and also to simulate standard machines running unmodified operating systems. This way we avoid the error-prone effort of developing drivers and libraries. The environment we propose in this paper enables us to show some of the difficulties we found when integrating diverse tools, and how we were able to overcome them. Furthermore we show some important details to have into account in order to do a valid full-system simulation of multicomputers, mostly related with synchronization and timing. Thus, a trade-off has to be found between simulation speed and accuracy of results.