Mixed state on a sparsely encoded associative memory model

 We analyzed symmetric mixed states corresponding to the so-called concept formation on a sparsely encoded associative memory model with 0–1 neurons. Three types of mixed states – OR, AND, and a majority decision – are described as typical examples. Each element of the OR mixed state is composed of corresponding memory pattern elements by means of the OR operation. The other two types are similarly defined. By analyzing their equilibrium properties through self-consistent signal-to-noise analysis and computer simulation, we found that the storage capacity of the OR mixed state diverges in a sparse limit, but that the other states do not diverge. In addition, we found that the optimal threshold values, which maximize the storage capacity for the memory pattern and the OR mixed state, coincide with each other in the spare limit. We conclude that the OR mixed state is a reasonable representative of mixed state in the sparse limit. Received: 10 November 1999 / Accepted in revised form: 5 April 2001     

Background-activity-dependent properties of a network model for working memory that incorporates cellular bistability

In models of working memory, transient stimuli are encoded by feature-selective persistent neural activity. Network models of working memory are also implicitly bistable. In the absence of a brief stimulus, only spontaneous, low-level, and presumably nonpatterned neural activity is seen. In many working-memory models, local recurrent excitation combined with long-range inhibition (Mexican hat coupling) can result in a network-induced, spatially localized persistent activity or “bump state” that coexists with a stable uniform state. There is now renewed interest in the concept that individual neurons might have some intrinsic ability to sustain persistent activity without recurrent network interactions. A recent visuospatial working-memory model (Camperi and Wang 1998) incorporates both intrinsic bistability of individual neurons within a firing rate network model and a single population of neurons on a ring with lateral inhibitory coupling. We have explored this model in more detail and have characterized the response properties with changes in background synaptic input I_o and stimulus width. We find that only a small range of I_o yields a working-memory-like coexistence of bump and uniform solutions that are both stable. There is a rather larger range where only the bump solution is stable that might correspond instead to a feature-selective long-term memory. Such a network therefore requires careful tuning to exhibit working-memory-like function. Interestingly, where bumps and uniform stable states coexist, we find a continuous family of stable bumps representing stimulus width. Thus, in the range of parameters corresponding to working memory, the model is capable of capturing a two-parameter family of stimulus features including both orientation and width.     

Neural coding for the retrieval of multiple memory patterns

We investigate the retrieval dynamics in a feature-based semantic memory model, in which the features are coded by neurons of the Hindmarsh–Rose type in the chaotic regime. We consider the retrieval process as consisting of the synchronized firing activity of the neurons coding for the same memory pattern. The retrieval dynamics is investigated for multiple patterns, with particular attention to the case of overlapping memories. In this case, we hypothesize a dynamical nontransitive mechanism based on synchronization, that allows for a shared feature to participate in multiple memory representations. The problem of the choice of a cognitive plausible time-scale for the retrieval analysis is investigated by analyzing the information that can be inferred from finite-time analyses. Different types of indicators are proposed in order to evaluate the temporal dynamics of the neurons engaged in the retrieval process. We interpret the simulation results as suggestive of a role for chaotic dynamics in allowing for flexible composition of elementary meaningful units in memory representations.     

A model of cell firing patterns during epileptic seizures

A simplified model is presented of the dynamics of excitatory and inhibitory neurons in the cerebral cortex. A key feature of the model is that neurons may cease to fire when strongly depolarized (spike inactivation). Computer simulations for different parameters reveal five classes of solutons: a) steady states in which neither excitatory nor inhibitory cells are active, b) steady states in which one or both types of cells fire repetitively, c) states in which one type of cell fluctuates rapidly between bursts of action potentials and inactivity due to strong depolarization, d) rhythmic activity in which both types of cells fire in unison followed by a period of spike inactivation and e) states similar to d but in which the inhibitory cells never produce action potentials. Solutions b, c, d, and e qualitatively resemble the different firing patterns observed during experimental seizures. It is shown that changes in those parameters that are functions of potassium concentration can induce changes in the type of solution. It is therefore proposed that the increase in extracellular potassium concentration during seizures may be responsible for the progressive changes observed in firing patterns and particularly for the transition from tonic to clonic patterns. A method is also outlined for testing the predictions of the model.     

The storage and recall of auditory memory

The architecture of the auditory memory is investigated. The auditory information is assumed to be represented by f–t patterns. With the help of a psycho-physical experiment it is demonstrated that the storage of these patterns is highly folded in the sense that a long signal is broken into many short stretches before being stored in the memory. Recognition takes place by correlating newly heard input in the short term memory to information previously stored in the long term memory. We show that this correlation is performed after the input is accumulated and held statically in the short term memory.     

Short-term memory trace in rapidly adapting synapses of inferior temporal cortex

Visual short-term memory tasks depend upon both the inferior temporal cortex (ITC) and the prefrontal cortex (PFC). Activity in some neurons persists after the first (sample) stimulus is shown. This delay-period activity has been proposed as an important mechanism for working memory. In ITC neurons, intervening (nonmatching) stimuli wipe out the delay-period activity; hence, the role of ITC in memory must depend upon a different mechanism. Here, we look for a possible mechanism by contrasting memory effects in two architectonically different parts of ITC: area TE and the perirhinal cortex. We found that a large proportion (80%) of stimulus-selective neurons in area TE of macaque ITCs exhibit a memory effect during the stimulus interval. During a sequential delayed matching-to-sample task (DMS), the noise in the neuronal response to the test image was correlated with the noise in the neuronal response to the sample image. Neurons in perirhinal cortex did not show this correlation. These results led us to hypothesize that area TE contributes to short-term memory by acting as a matched filter. When the sample image appears, each TE neuron captures a static copy of its inputs by rapidly adjusting its synaptic weights to match the strength of their individual inputs. Input signals from subsequent images are multiplied by those synaptic weights, thereby computing a measure of the correlation between the past and present inputs. The total activity in area TE is sufficient to quantify the similarity between the two images. This matched filter theory provides an explanation of what is remembered, where the trace is stored, and how comparison is done across time, all without requiring delay period activity. Simulations of a matched filter model match the experimental results, suggesting that area TE neurons store a synaptic memory trace during short-term visual memory.     

Associative memory in quaternionic Hopfield neural network

Associative memory networks based on quaternionic Hopfield neural network are investigated in this paper. These networks are composed of quaternionic neurons, and input, output, threshold, and connection weights are represented in quaternions, which is a class of hypercomplex number systems. The energy function of the network and the Hebbian rule for embedding patterns are introduced. The stable states and their basins are explored for the networks with three neurons and four neurons. It is clarified that there exist at most 16 stable states, called multiplet components, as the degenerated stored patterns, and each of these states has its basin in the quaternionic networks.     

Neural coding for the retrieval of multiple memory patterns

We investigate the retrieval dynamics in a feature-based semantic memory model, in which the features are coded by neurons of the Hindmarsh-Rose type in the chaotic regime. We consider the retrieval process as consisting of the synchronized firing activity of the neurons coding for the same memory pattern. The retrieval dynamics is investigated for multiple patterns, with particular attention to the case of overlapping memories. In this case, we hypothesize a dynamical nontransitive mechanism based on synchronization, that allows for a shared feature to participate in multiple memory representations. The problem of the choice of a cognitive plausible time-scale for the retrieval analysis is investigated by analyzing the information that can be inferred from finite-time analyses. Different types of indicators are proposed in order to evaluate the temporal dynamics of the neurons engaged in the retrieval process. We interpret the simulation results as suggestive of a role for chaotic dynamics in allowing for flexible composition of elementary meaningful units in memory representations.     

Neuronal Mechanisms Encoding Global-to-Fine Information in Inferior-Temporal Cortex

Sugase et al. found that global information is represented at the initial transient firing of a single face-responsive neuron in inferior-temporal (IT) cortex, and that finer information is represented at the subsequent sustained firing. A feed-forward model and an attractor network are conceivable models to reproduce this dynamics. The attractor network, specifically an associative memory model, is employed to elucidate the neuronal mechanisms producing the dynamics. The results obtained by computer simulations show that a state of neuronal population initially approaches to a mean state of similar memory patterns, and that it finally converges to a memory pattern. This dynamics qualitatively coincides with that of face-responsive neurons. The dynamics of a single neuron in the model also coincides with that of a single face-responsive neuron. Furthermore, we propose two physiological experiments and predict the results from our model. Both predicted results are not explainable by the feed-forward model. Therefore, if the results obtained by actual physiological experiments coincide with our predicted results, the attractor network might be the neuronal mechanisms producing the dynamics of face-responsive neurons.     

Modeling monkeys: A comparison of computer-generated and naturally occurring foraging patterns in two species of neotropical primates

We present a series of computer-generated foraging models (random movement, olfactory navigation, and spatial memory) designed to examine the manner in which sensory cues and cognitive skills might be used by rainforest monkeys to locate patchily distributed feeding sites. These simulations are compared with data collected in the Amazon Basin of northeastern Peru on the foraging patterns of two species of neotropical primates, the moustached tamarin monkey (Saguinus mystax) and the saddle-back tamarin monkey (Saguinus fuscicollis). The results indicate that, although tamarins may rely on olfactory cues to locate nearby feeding sites, their foraging patterns are better explained by an ability to maintain a detailed spatial map of the location and distribution of hundreds of feeding trees in their home range. There is evidence that such informationis retained for a period of at least several weeks and is used to minimize the distance traveled between widely scattered feeding sites. The use of computer simulations provides a powerful research tool for generating predictive models regarding the role of memory and sensory cues in animal foraging patterns.     

Cross-talk theory of memory capacity in neural networks

The present paper presents a theory for the mechanics of cross-talk among constituent neurons in networks in which multiple memory traces have been embedded, and develops criteria for memory capacity based on the disruptive influences of this cross-talk. The theory is based on interconnection patterns defined by the sequential configuration model of dynamic firing patterns. The theory accurately predicts the memory capacities observed in computer simulated nets, and predicts that cortical-like modules should be able to store up to about 300–900 selectively retrievable memory traces before disruption by cross-talk is likely. It also predicts that the cortex may has designed itself for modules of 30,000 neurons to at least in part to optimize memory capacity.     

An interpretation of 1/f fluctuations in neuronal spike trains during dream sleep

The mesencephalic reticular formation (MRF) neurons are regarded as contributing to the activation of the cerebral cortex. We have investigated the statistical characteristics of the single neuronal activity in the MRF of cat during two activated states: paradoxical sleep (PS) and state in which the animal is watching birds (BW). 1/f-like spectra are observed for both PS and BW states, being more pronounced for PS state. For the interpretation of these findings, we have applied the clustering Poisson process, which not only gives rise to a 1/f spectrum but also suggests a generation mechanism. The MRF neuronal activities in PS and BW are closely fitted by the clustering Poisson process, both in terms of power spectral density and counting statistics. These results strongly suggest that the activities of MRF neurons in PS and BW can be interpreted as the superpositions of randomly occurring clusters which consist of various number of impulses.     

Properties of human visual memory for block patterns

Several characteristics of human short-term visual memory (STVM) were specified through a series of experiments, by using block patters (BPs) of varying complexity and matrix size (n-by-n). For each matrix size, BPs with high and low complexity were formed (i.e.n-by-n-H andn-by-n-L). In experiment I, the characteristics of the acquisition process were examined through a recall task. The recall rate for a single glance (exposure time less than 0.3s) is more than 90% for 3-by-3 and 4-by-4-L BPs. For 4-by-4-H BPs, an improvement in recall rate was not found even when exposure time was increased to 2.4s. The recall rate for 6-by-6-H, 7-by-7, and 8-by-8 BPs did not change even when the exposure time was increased to 9s. In experiment II, the characteristics of the STVM decay process were examined using a recall task. Though a difference between the 4-by-4-L and 4-by-4-H acquisition rates was found, no difference was found in the forgetting rates. No decay was found for 6-by-6 BPs. Furthermore, the information obtained during a short duration was not forgotten for 4-by-4, and 6-by-6 BPs. It was concluded from these results that:1) The acquisition rate into STVM depends upon figural complexity.2) The decay rate does not depend upon figural complexity.3) The limit of STVM was between 4-by-4-L, and 4-by-4-H BPs.4) The recall performance for 6-by-6 BPs reflects the information stored in long-term visual memory. Although the acquisition rate into STVM depend upon figural complexity, it appeared in experiment IV that the number of subpatterns into which subjects segmented BPs when memorizing them was highly correlated with rated figural complexity. It also appeared that the number of memory chunks estimated from the data of interrecall-interval was not correlated with the complexity. Finally, a process model for visual memory for block patterns was proposed.     

Effects of neuromodulation in a cortical network model of object working memory dominated by recurrent inhibition

Experimental evidence suggests that the maintenance of an item in working memory is achieved through persistent activity in selective neural assemblies of the cortex. To understand the mechanisms underlying this phenomenon, it is essential to investigate how persistent activity is affected by external inputs or neuromodulation. We have addressed these questions using a recurrent network model of object working memory. Recurrence is dominated by inhibition, although persistent activity is generated through recurrent excitation in small subsets of excitatory neurons.Our main findings are as follows. (1) Because of the strong feedback inhibition, persistent activity shows an inverted U shape as a function of increased external drive to the network. (2) A transient external excitation can switch off a network from a selective persistent state to its spontaneous state. (3) The maintenance of the sample stimulus in working memory is not affected by intervening stimuli (distractors) during the delay period, provided the stimulation intensity is not large. On the other hand, if stimulation intensity is large enough, distractors disrupt sample-related persistent activity, and the network is able to maintain a memory only of the last shown stimulus. (4) A concerted modulation of GABA _A and NMDA conductances leads to a decrease of spontaneous activity but an increase of persistent activity; the enhanced signal-to-noise ratio is shown to increase the resistance of the network to distractors. (5) Two mechanisms are identified that produce an inverted U shaped dependence of persistent activity on modulation. The present study therefore points to several mechanisms that enhance the signal-to-noise ratio in working memory states. These mechanisms could be implemented in the prefrontal cortex by dopaminergic projections from the midbrain.     

Capacity analysis in multi-state synaptic models: a retrieval probability perspective

We define the memory capacity of networks of binary neurons with finite-state synapses in terms of retrieval probabilities of learned patterns under standard asynchronous dynamics with a predetermined threshold. The threshold is set to control the proportion of non-selective neurons that fire. An optimal inhibition level is chosen to stabilize network behavior. For any local learning rule we provide a computationally efficient and highly accurate approximation to the retrieval probability of a pattern as a function of its age. The method is applied to the sequential models (Fusi and Abbott, Nat Neurosci 10:485–493, 2007) and meta-plasticity models (Fusi et al., Neuron 45(4):599–611, 2005; Leibold and Kempter, Cereb Cortex 18:67–77, 2008). We show that as the number of synaptic states increases, the capacity, as defined here, either plateaus or decreases. In the few cases where multi-state models exceed the capacity of binary synapse models the improvement is small.     

Reliable short-term memory in the trion model: toward a cortical language and grammar

 It has previously been shown that Hebb learning in a single column in the trion model of cortical organization occurs by selection. Motivated by von Neumann's solution for obtaining reliability and by models of circulating cortical activity, we introduce Hebb intercolumnar couplings to achieve dramatic enhancements in reliability in the firing of connected columns. In order for these enhancements to occur, specific temporal phase differences must exist between the same inherent spatial-temporal memory patterns in connected columns. We then generalize the criteria of large enhancements in the global firing of the entire connected columnar network to investigate the case when different inherent memory patterns are in the columns. The spatial rotations as well as the temporal phases now are crucial. Only certain combinations of inherent memory patterns meet these criteria with the symmetry properties playing a major role. The columnar order of these memory patterns not in the same symmetry family can be extremely important. This yields the first higher-level architecture of a cortical language and grammar within the trion model. The implications of this result with regard to an innate human language and grammar are discussed. Received: 14 June 2000 / Accepted in revised form: 25 July 2000     

Macrophage ion currents are fit by a fractional model and therefore are a time series with memory

We studied macroscopic ion currents from macrophages and compared their patterns of behavior using classical and fractal analysis. Peak and steady state currents were measured respectively at the beginning and end of a voltage-clamp pulse. Hurst coefficients H and fractional dimensions were calculated for the current fluctuations (I _H ) during the intervening interval; these fluctuations are usually assumed to be white noise. We show that I _H is different from 0.5 and that the increments are stationary, indicating that the dynamic model has memory and that the intervening current fluctuations cannot be considered as white noise. I _H is less than 0.5, implying an antipersistent pattern. In addition, we show that the relation between inactivation and I _H versus voltage V fit an equation I _H (V) = f(V, α, m, d), where α is associated with fractional calculus and m and d are free parameters. Fitting by a fractional model confirms that the phenomenon has memory.     

Why some tits store food and others do not: evaluation of ecological factors

Paridae are among the bird families benefiting from food storing. However, not all its members hoard food. Our objective was to clarify the role of ecological factors in occurrence of food storing. We reviewed the data on major ecological characteristics of the Paridae species and analysed their association with the presence/absence of food storing. Our statistical model revealed that geography (distribution in North America) and taxonomy (genus Poecile) are better predictors of food storing than any of the studied ecological traits. Nevertheless, food-storing Poecile species inhabit mixed or coniferous woodlands with seasonal richness of food, while non-storing species tend to prefer edge and open habitats, where alternative food sources are available. Sociality and territoriality outside breeding season coincides with food storing. The analysis performed within the Baeolophus–Lophophanes–Periparus–Poecile clade with ancestral food storing revealed no factor except continental climate that would explain the persistence of food storing. The phylogenetic analysis of ancestral states of the studied characters allowed us to propose a possible scenario for the emergence of food storing in Paridae. (1) Food storing is not ancestral in Paridae and appeared only once in the common ancestor of the Baeolophus–Lophophanes–Periparus–Poecile group. (2) According to estimation of molecular clock in Paridae, food storing appeared before their radiation in North America.     

A geometric synthesis method to realize a switching function

Frequently in threshold logic it is necessary to realize a switching function f of n variables in a two or more stages realization, both because it is not possible to realize f in a single gate and for reliability considerations. This means that the given function must be decomposed into a certain number of threshold functions, each of them having a number of variables less than n. When a geometric representation of switching functions is considered, the problem is equivalent to that of looking for functions whose patterns (on-sets) cover, in an adequate way, the on-set of the function f to be realized. In this paper a heuristic covering approach has been introduced which provides a manageable method for searching for a two stage realization of f. Furthermore a class of switching functions whose patterns possess a particular geometric shape (shell structure) has been considered. It has been proved that such functions (shell functions) are linearly separable and that one of their separating systems strictly depends on the centre and on the diameter of the shell structure. Gates implementing shell functions have been considered and their use has been revealed more convenient than that of majority gates in the solution of the problem of network synthesis. To clarify this advantage an example has been discussed in detail.     

Roles of dynamic linkage of stable attractors across cortical networks in recalling long-term memory

 We propose a neural network model for a category-association task. By simulating the model, neuronal relevance of cortical interactions to recalling long-term memory was investigated. The model consists of the left and right hemispheres, each of which has IT (inferotemporal cortex) and PC (prefrontal cortex) networks. Information about visual features and their categories were encoded into point attractors of the IT and PC networks, respectively. In the task, the IT network of the right hemisphere was stimulated with a cue feature. After a delay period, the IT network of the left hemisphere was simultaneously stimulated with the choice feature and an irrelevant feature. The cue and choice features belong to the same category, while the irrelevant feature belongs to another category. To complete the task, the IT network must select the point attractor corresponding to the choice feature. We demonstrate that the top-down pathway (PC-to-IT) triggers the retrieval of long-term memory of the choice feature from the IT, and the bottom-up pathway (IT-to-PC) contributes to the maintenance of the retrieved memory during the delay period. The key mechanism for the retrieval and maintenance of that memory is the dynamic linkage of attractors across separate cortical networks. We show that a single hemisphere is sufficient for the memory retrieval, but it is advantageous to use the two hemispheres because the retrieved memory is thereby retained with greater reliability until the brain chooses the choice feature. Received: 4 April 2001 / Accepted in revised form: 17 September 2002 / Published online: 20 January 2003 Correspondence to: O. Hoshino (e-mail: hoshino@cc.oita-u.ac.jp, Tel.: +81-97-554-7301, Fax: +81-97-554-7507)