Visual associative memory and the orientation-contingent color after-effect

The traditional explanation of the McCollough effect (ME) by selective adaptation of single detectors selective to color and orientation suffers from a number of inconsistencies: 1) the ME lasts much longer (from several days up to 3 months) than the ordinary adaptation, the decay of the effect being completely arrested by night sleep or occluding the eye for a long time; 2) the strength of the ME practically does not depend on the intensity of adapting light; and 3) a set of related pattern-contingent after-effects discovered later required for such an explanation new detectors, specific for other patterns. These properties can be explained, however, in the framework of associative memory and novelty filters. A computational model has been developed, which consists of 1) an input layer of two (left and right eyes) square matrices with two analog receptors (red and green) in each pixel, 2) an isomorphic associative neural layer, each analog neuron being synaptically connected with all receptors of both eyes, and 3) an output layer (novelty filter). The modification of synaptic efficacies conforms to the Hebb learning rule. The function of the model was examined by simulation. After a few presentations of colored gratings, the model displays the ME that is slowly destroyed by subsequent presentations of random pictures. With a sufficiently large receptor matrix, the effect lasts a thousand times longer than the period of adaptation. Continuous darkness does not change the strength of the effect. Like in real ME, the model does not display interocular transfer. The model can account for different pattern-contingent color after-effects without assuming any predetermined specific detectors. Such detectors are constructed in the course of adaptation to specific stimuli (gratings).     

Visual homing: an insect perspective

The ability to learn the location of places in the world and to revisit them repeatedly is crucial for all aspects of animal life on earth. It underpins animal foraging, predator avoidance, territoriality, mating, nest construction and parental care. Much theoretical and experimental progress has recently been made in identifying the sensory cues and the computational mechanisms that allow insects (and robots) to find their way back to places, while the neurobiological mechanisms underlying navigational abilities are beginning to be unravelled in vertebrate and invertebrate models. Studying visual homing in insects is interesting, because they allow experimentation and view-reconstruction under natural conditions, because they are likely to have evolved parsimonious, yet robust solutions to the homing problem and because they force us to consider the viewpoint of navigating animals, including their sensory and computational capacities.     

Improving recall from an associative memory

The associative net as a model of biological associative memory is investigated. Calculating the output pattern retrieved from a partially connected associative net presented with noisy input cues involves several computations. This is complicated by variations in the dendritic sums of the output units due to errors in the cue and differences in input activity and unit usage. The possible implementation of these computations by biological neural machinery is unclear. We demonstrate that a relatively simple transformation can reduce variation in the dendritic sums. This leads to a winners-take-all type of strategy that produces increased recall performance which is equivalent to the more complicated optimal strategy proposed by others. We describe in detail the possible biological implications of our strategies, the novel feature of which ascribes a role to the NMDA and non-NMDA channels found in hippocampal pyramidal cells. Received: 13 April 1994 / Accepted: 25 October 1994     

Construction of an associative memory using unstable periodic orbits of a chaotic attractor

Unstable periodic orbits are the skeleton of a chaotic attractor. We constructed an associative memory based on the chaotic attractor of an artificial neural network, which associates input patterns to unstable periodic orbits. By processing an input, the system is driven out of the ground state to one of the pre-defined disjunctive areas of the attractor. Each of these areas is associated with a different unstable periodic orbit. We call an input pattern learned if the control mechanism keeps the system on the unstable periodic orbit during the response. Otherwise, the system relaxes back to the ground state on a chaotic trajectory. The major benefits of this memory device are its high capacity and low-energy consumption. In addition, new information can be simply added by linking a new input to a new unstable periodic orbit.     

On associative memory

The information storing capacity of certain associative and auto-associative memories is calculated. For example, in a 100×100 matrix of 1 bit storage elements more than 6,500 bits can be stored associatively, and more than 688,000 bits in a 1,000×1,000 matrix. Asymptotically, the storage capacity of an associative memory increases proportionally to the number of storage elements. The usefulness of associative memories, as opposed to conventional listing memories, is discussed — especially in connection with brain modelling.     

Molecular associative memory

An example of a kinetic system with address-bearing molecules and directed interactions is investigated. We show that by introducing exchange between Ising spins via the address-bearing messengers, the Hopfield model of associative memory can be made local.     

An algebraic model of an associative noise-like coding memory

A mathematical model of an associative memory is presented, sharing with the optical holography memory systems the properties which establish an analogy with biological memory. This memory system-developed from Gabor's model of memoryis based on a noise-like coding of the information by which it realizes a distributed, damage-tolerant, equipotential storage through simultaneous state changes of discrete substratum elements. Each two associated items being stored are coded by each other by means of two noise-like patterns obtained from them through a randomizing preprocessing. The algebraic braic transformations operating the information storage and retrieval are matrix-vector products involving Toeplitz type matrices. Several noise-like coded memory traces are superimposed on a common substratum without crosstalk interference; moreover, extraneous noise added to these memory traces does not injure the stored information. The main performances shown by this memory model are: i) the selective, complete recovering of stored information from incomplete keys, both mixed with extraneous information and translated from the position learnt; ii) a dynamic recollection where the information just recovered acts as a new key for a sequential retrieval process; iii) context-dependent responses. The hypothesis that the information is stored in the nervous system through a noise-like coding is suggested. The model has been simulated on a digital computer using bidimensional images.     

Visual homing in analog hardware

Insects of several species rely on visual landmarks for returning to important locations in their environment. The "average landmark vector model" is a parsimonious model which reproduces some aspects of the visual homing behavior of bees and ants. To gain insights in the structure and complexity of the neural apparatus that might underly the navigational capabilities of these animals, the average landmark vector model was implemented in analog hardware and used to control a mobile robot. The experiments demonstrate that the apparently complex task of visual homing might be realized by simple and mostly peripheral neural circuits in insect brains.     

Error tolerant associative memory

We present a new approach to enlarging the basin of attraction of associative memory, including auto-associative memory and temporal associative memory. The memory trained by means of this method can tolerate and recover from seriously noisy patterns. Simulations show that this approach will greatly reduce the number of limit cycles.     

Generalized protein tertiary structure recognition using associative memory Hamiltonians.

In previous papers, a method of protein tertiary structure recognition was described based on the construction of an associative memory Hamiltonian, which encoded the amino acid sequence and the C alpha co-ordinates of a set of database proteins. Using molecular dynamics with simulated annealing, the ability of the Hamiltonian to successfully recall the structure of a protein in the memory database was successfully demonstrated, as long as the total number of database proteins did not exceed a characteristic value, called the capacity of the Hamiltonian, equal to 0.5N to 0.7N, where N is the number of amino acid residues in the protein to be recalled. In this paper, we describe the development of additional methods to increase the capacity of the Hamiltonian, including use of a more complete representation of the protein backbone and the incorporation of contextual information into the Hamiltonian through the use of secondary structure prediction. In addition, we further extend the ability of associative memory models to predict the tertiary structures of proteins not present in the protein data set, by making the Hamiltonian invariant with respect to biological symmetries that represent site mutations and insertions and deletions. The ability of the Hamiltonian to generalize from homologous proteins to an unknown protein in the presence of other unrelated proteins in the data set is demonstrated.     

A cognitive and associative memory

By introducing a physiological constraint in the auto-correlation matrix memory, the system is found to acquire an ability in cognition i.e. the ability to identify and input pattern by its proximity to any one of the stored memories. The physiological constraint here is that the attribute of a given synapse (i.e. excitatory or inhibitory) is uniquely determined by the neuron it belongs. Thus the synaptic coupling is generally not symmetric. Analytical and numerical analyses revealed that the present model retrieves a memory if an input pattern is close to the pattern of the stored memories; if not, it gives a clear response by going into a special mode where almost all neurons are in the same state in each time step. This uniform mode may be stationary or periodic, depending on whether or not the number of the excitatory neurons exceeds the number of inhibitory neurons.     

Internal representations for associative memory

We describe a class of feed forward neural network models for associative content addressable memory (ACAM) which utilize sparse internal representations for stored data. In addition to the input and output layers, our networks incorporate an intermediate processing layer which serves to label each stored memory and to perform error correction and association. We study two classes of internal label representations: the unary representation and various sparse, distributed representations. Finally, we consider storage of sparse data and sparsification of data. These models are found to have advantages in terms of storage capacity, hardware efficiency, and recall reliability when compared to the Hopfield model, and to possess analogies to both biological neural networks and standard digital computer memories.     

A holographic model for associative memory chains

Holographic brain models are well suited to describe specific brain functions. Central nervous systems and holographic systems both show parallel information processing and non-localized storage in common. To process information both systems use correlation functions suggesting to develop cybernetical brain models in terms of holography. Associative holographic storage is done with two simultaneously existing patterns. They may reconstruct each other mutually. Time-sequentially existing patterns are connected to associative chains, if every two succeeding patterns do exist within a common period of time in order to be stored in pairs. Read out (recall) of associative chains—reconstructing coupled patterns which didn't exist simultaneously—requires advanced holographic techniques. Three different methods are described and tested experimentally. The underlying principles are feedback mechanisms, nonlinearities of the storage material and tridimensional architecture of the voluminous recording medium. Those principles evidently occur in neural storage systems supporting analogous information processing in neural- and holographic systems.     

Pseudo-relaxation learning algorithm for complex-valued associative memory

HAM (Hopfield Associative Memory) and BAM (Bidirectinal Associative Memory) are representative associative memories by neural networks. The storage capacity by the Hebb rule, which is often used, is extremely low. In order to improve it, some learning methods, for example, pseudo-inverse matrix learning and gradient descent learning, have been introduced. Oh introduced pseudo-relaxation learning algorithm to HAM and BAM. In order to accelerate it, Hattori proposed quick learning. Noest proposed CAM (Complex-valued Associative Memory), which is complex-valued HAM. The storage capacity of CAM by the Hebb rule is also extremely low. Pseudo-inverse matrix learning and gradient descent learning have already been generalized to CAM. In this paper, we apply pseudo-relaxation learning algorithm to CAM in order to improve the capacity.     

Stability and attractivity in associative memory networks

We focus on stable and attractive states in a network having two-state neuron-like elements. We calculate the connection matrix which guarantees the stability and the strongest attractivity of p memorized patterns. We present an analytical evaluation of the patterns' attractivity. These results are illustrated by some computer simulations.     

An after-shannon measure of the storage capacity of an associative noise-like coding memory

The maximum amount of information that can be stored, on the average, in each storage element, according to an associative scheme, has been measured for the memory model proposed by the author (Bottini 1980). In this model, the (binary) items being stored are coded by noise-like keys and the memory traces formed in this way are superimposed, by algebraic addition, on the same many-level storage elements. It is shown that the problem of measuring the information retrieved from the memory in a single recall and the problem — concerning the data-communication field —of measuring the information transmitted over a noisy channel are formally similar. In particular, the Shannon noisy-channel coding theorem can find an application also in our case of an associative memory. Finally, it is evidenced that the so-called matrix model of an associative memory has the same storage capacity as the model studied here.     

On the storage capacity of an associative memory with randomly distributed storage elements

In an associative memory with randomly distributed storage elements at least 0.05 bit per storage element can be stored.     

Cytokine control of memory B cell homing machinery

The germinal center (GC) is a pivotal site for the development of B cell memory. Whereas GC B cells do not chemotax to most chemokines and do not express the adhesion receptors L-selectin, alpha(4)beta(7), and cutaneous lymphocyte Ag (CLA), memory B cells respond to various chemotactic signals and express adhesion receptors. In this study, we show that CD40 ligand, IL-2, and IL-10 together drive this transition of GC B cells to memory phenotype in vitro, up-regulating memory B cell markers, chemotactic responses to CXC ligand (CXCL)12, CXCL13, and CCL19, and expression of adhesion receptors L-selectin, alpha(4)beta(7), and CLA. Moreover, addition of IL-4 modulates this transition, preventing chemotactic responses to CXCL12 and CXCL13 (but not to CCL19), and inhibiting the re-expression of L-selectin, but not of CLA or alpha(4)beta(7). CCR7 expression, responsiveness to CCL19, and L-selectin/alpha(4)beta(7) phenotype are coordinately regulated. Thus, IL-2/IL-10 and IL-4 play important and distinctive roles in developing the migratory capacities of memory B cells.     

Visual motion: homing in on small target detectors

Tracking moving targets is essential for animals that pursue prey or conspecifics. Recent studies in male and female hoverflies have described classes of neurons that detect the movements of small targets against a moving background but the mechanisms generating their responses remain unclear.