Synaptic efficacy shapes resource limitations in working memory

Working memory (WM) is limited in its temporal length and capacity. Classic conceptions of WM capacity assume the system possesses a finite number of slots, but recent evidence suggests WM may be a continuous resource. Resource models typically assume there is no hard upper bound on the number of items that can be stored, but WM fidelity decreases with the number of items. We analyze a neural field model of multi-item WM that associates each item with the location of a bump in a finite spatial domain, considering items that span a one-dimensional continuous feature space. Our analysis relates the neural architecture of the network to accumulated errors and capacity limitations arising during the delay period of a multi-item WM task. Networks with stronger synapses support wider bumps that interact more, whereas networks with weaker synapses support narrower bumps that are more susceptible to noise perturbations. There is an optimal synaptic strength that both limits bump interaction events and the effects of noise perturbations. This optimum shifts to weaker synapses as the number of items stored in the network is increased. Our model not only provides a circuit-based explanation for WM capacity, but also speaks to how capacity relates to the arrangement of stored items in a feature space.     

Background-activity-dependent properties of a network model for working memory that incorporates cellular bistability

In models of working memory, transient stimuli are encoded by feature-selective persistent neural activity. Network models of working memory are also implicitly bistable. In the absence of a brief stimulus, only spontaneous, low-level, and presumably nonpatterned neural activity is seen. In many working-memory models, local recurrent excitation combined with long-range inhibition (Mexican hat coupling) can result in a network-induced, spatially localized persistent activity or “bump state” that coexists with a stable uniform state. There is now renewed interest in the concept that individual neurons might have some intrinsic ability to sustain persistent activity without recurrent network interactions. A recent visuospatial working-memory model (Camperi and Wang 1998) incorporates both intrinsic bistability of individual neurons within a firing rate network model and a single population of neurons on a ring with lateral inhibitory coupling. We have explored this model in more detail and have characterized the response properties with changes in background synaptic input I_o and stimulus width. We find that only a small range of I_o yields a working-memory-like coexistence of bump and uniform solutions that are both stable. There is a rather larger range where only the bump solution is stable that might correspond instead to a feature-selective long-term memory. Such a network therefore requires careful tuning to exhibit working-memory-like function. Interestingly, where bumps and uniform stable states coexist, we find a continuous family of stable bumps representing stimulus width. Thus, in the range of parameters corresponding to working memory, the model is capable of capturing a two-parameter family of stimulus features including both orientation and width.     

Two components of the receptor current are developed from distinct elementary signals in Limulus ventral nerve photoreceptor

Transient elementary currents, bumps, stimulated by short dim light flashes were measured in ventral nerve photoreceptors of Limulus. It is demonstrated that light activates two types of bumps, which form two distinct components of the receptor current at higher light intensities. The two bump types, which are both assumed to be activated by single absorbed photons, differ in current amplitude and kinetic parameters. The current amplitude of one bump type is smaller than 0.3 nA and that of the other type is in the usual current range of up to several nanoamperes. The average latency of small bumps measured from the short stimulus flash is shorter than that of the large bumps. The small bumps have slower activation kinetics than the large bumps. It is demonstrated that with increasing flash intensity the small bumps overlap first and form a macroscopic current, on top of which the large bumps are superimposed. Results indicate that a single absorbed photon selectively activates only one kind of the enzyme cascades evoking one bump type. We conclude that the active meta conformation of a rhodopsin molecule selectively binds a specific type of G-protein, which is involved in the stimulation of one of the transduction cascades. The two bump types, which are the elements of two macroscopic current components support the previous assumption that light activates different transduction mechanisms in Limulus photoreceptors.     

Place Cell Rate Remapping by CA3 Recurrent Collaterals

Episodic-like memory is thought to be supported by attractor dynamics in the hippocampus. A possible neural substrate for this memory mechanism is rate remapping, in which the spatial map of place cells encodes contextual information through firing rate variability. To test whether memories are stored as multimodal attractors in populations of place cells, recent experiments morphed one familiar context into another while observing the responses of CA3 cell ensembles. Average population activity in CA3 was reported to transition gradually rather than abruptly from one familiar context to the next, suggesting a lack of attractive forces associated with the two stored representations. On the other hand, individual CA3 cells showed a mix of gradual and abrupt transitions at different points along the morph sequence, and some displayed hysteresis which is a signature of attractor dynamics. To understand whether these seemingly conflicting results are commensurate with attractor network theory, we developed a neural network model of the CA3 with attractors for both position and discrete contexts. We found that for memories stored in overlapping neural ensembles within a single spatial map, position-dependent context attractors made transitions at different points along the morph sequence. Smooth transition curves arose from averaging across the population, while a heterogeneous set of responses was observed on the single unit level. In contrast, orthogonal memories led to abrupt and coherent transitions on both population and single unit levels as experimentally observed when remapping between two independent spatial maps. Strong recurrent feedback entailed a hysteretic effect on the network which diminished with the amount of overlap in the stored memories. These results suggest that context-dependent memory can be supported by overlapping local attractors within a spatial map of CA3 place cells. Similar mechanisms for context-dependent memory may also be found in other regions of the cerebral cortex.     

Differential calcium sensitivity of two types of quantum bumps in Limulus ventral nerve photoreceptors

Ventral photoreceptors in Limulus polyphemus were continuously illuminated with dim light and produced two types of quantum bumps. Bumps of one type had a relatively symmetrical shape and fast rise and decay kinetics. In addition, bumps were detected with a small amplitude and slower kinetics. We termed these bump types C₂ and C₁ bumps, respectively. The half width of a bump divided by its amplitude was found to be a reliable criterion to distinguish between the bump types. This shape quotient is smaller than 0.5 ms/pA for the C₂ bumps and larger than 1 ms/pA for the C₁ bumps. Lowering the extracellular calcium concentration from 10 mM to 0.25 mM caused an increase in the average amplitude of C₂ bumps to 196%. After the injection of small amounts of 1.2 mM EGTA solution the amplitude of these bumps was reduced to about 30 and 50% in two cells studied. By contrast, in both cases C₁ bump amplitudes were not affected significantly. We conclude that the two bump types are generated in one photoreceptor by the activation of different transduction pathways. These pathways are differentially sensitive to changes of the cytosolic calcium concentration.     

Persistent activity in neural networks with dynamic synapses

Persistent activity states (attractors), observed in several neocortical areas after the removal of a sensory stimulus, are believed to be the neuronal basis of working memory. One of the possible mechanisms that can underlie persistent activity is recurrent excitation mediated by intracortical synaptic connections. A recent experimental study revealed that connections between pyramidal cells in prefrontal cortex exhibit various degrees of synaptic depression and facilitation. Here we analyze the effect of synaptic dynamics on the emergence and persistence of attractor states in interconnected neural networks. We show that different combinations of synaptic depression and facilitation result in qualitatively different network dynamics with respect to the emergence of the attractor states. This analysis raises the possibility that the framework of attractor neural networks can be extended to represent time-dependent stimuli.     

Robustness and information propagation in attractors of random boolean networks

Attractors represent the long-term behaviors of Random Boolean Networks. We study how the amount of information propagated between the nodes when on an attractor, as quantified by the average pairwise mutual information ([Formula: see text]), relates to the robustness of the attractor to perturbations ([Formula: see text]). We find that the dynamical regime of the network affects the relationship between [Formula: see text] and [Formula: see text]. In the ordered and chaotic regimes, [Formula: see text] is anti-correlated with [Formula: see text], implying that attractors that are highly robust to perturbations have necessarily limited information propagation. Between order and chaos (for so-called "critical" networks) these quantities are uncorrelated. Finite size effects cause this behavior to be visible for a range of networks, from having a sensitivity of 1 to the point where [Formula: see text] is maximized. In this region, the two quantities are weakly correlated and attractors can be almost arbitrarily robust to perturbations without restricting the propagation of information in the network.     

Sensory feedback in a bump attractor model of path integration

Mammalian spatial navigation systems utilize several different sensory information channels. This information is converted into a neural code that represents the animal’s current position in space by engaging place cell, grid cell, and head direction cell networks. In particular, sensory landmark (allothetic) cues can be utilized in concert with an animal’s knowledge of its own velocity (idiothetic) cues to generate a more accurate representation of position than path integration provides on its own (Battaglia et al. The Journal of Neuroscience 24(19):4541–4550 (2004)). We develop a computational model that merges path integration with feedback from external sensory cues that provide a reliable representation of spatial position along an annular track. Starting with a continuous bump attractor model, we explore the impact of synaptic spatial asymmetry and heterogeneity, which disrupt the position code of the path integration process. We use asymptotic analysis to reduce the bump attractor model to a single scalar equation whose potential represents the impact of asymmetry and heterogeneity. Such imperfections cause errors to build up when the network performs path integration, but these errors can be corrected by an external control signal representing the effects of sensory cues. We demonstrate that there is an optimal strength and decay rate of the control signal when cues appear either periodically or randomly. A similar analysis is performed when errors in path integration arise from dynamic noise fluctuations. Again, there is an optimal strength and decay of discrete control that minimizes the path integration error.     

Unsupervised learning for robust working memory

Working memory is a core component of critical cognitive functions such as planning and decision-making. Persistent activity that lasts long after the stimulus offset has been considered a neural substrate for working memory. Attractor dynamics based on network interactions can successfully reproduce such persistent activity. However, it requires a fine-tuning of network connectivity, in particular, to form continuous attractors which were suggested for encoding continuous signals in working memory. Here, we investigate whether a specific form of synaptic plasticity rules can mitigate such tuning problems in two representative working memory models, namely, rate-coded and location-coded persistent activity. We consider two prominent types of plasticity rules, differential plasticity correcting the rapid activity changes and homeostatic plasticity regularizing the long-term average of activity, both of which have been proposed to fine-tune the weights in an unsupervised manner. Consistent with the findings of previous works, differential plasticity alone was enough to recover a graded-level persistent activity after perturbations in the connectivity. For the location-coded memory, differential plasticity could also recover persistent activity. However, its pattern can be irregular for different stimulus locations under slow learning speed or large perturbation in the connectivity. On the other hand, homeostatic plasticity shows a robust recovery of smooth spatial patterns under particular types of synaptic perturbations, such as perturbations in incoming synapses onto the entire or local populations. However, homeostatic plasticity was not effective against perturbations in outgoing synapses from local populations. Instead, combining it with differential plasticity recovers location-coded persistent activity for a broader range of perturbations, suggesting compensation between two plasticity rules.     

Dependence of bump rate and bump size in Limulus ventral nerve photoreceptor on light adaptation and calcium concentration

Bumps were recorded in Limulus ventral nerve photoreceptor as deflections in membrane voltage during 10 s illuminations by dim light which were repeated every 20 s. The bump amplitude vs frequency distribution and its dependence on the intensity of a preadapting light flash are described. Light adaptation which diminishes the average bump amplitude alters the character of the bump amplitude distribution from a curve with a convex region to a continuously falling concave curve. Weak light adaptation can increase frequency (and height) of the bumps elicited by constant stimuli. Raising the external Ca²⁺-concentration from 10 to 40 mmol/l augments the effect of a preadapting light flash in diminishing the bump amplitudes and also increases the bump frequency. The results are consistent with the assumptions

that light adaptation is based on a Ca²⁺-dependent reduction of the amplification factor which determines the bump size and

that the coupling between light induced rhodopsin reactions and bump generation is Ca²⁺-dependent.

     

Light-induced voltage noise in the photoreceptor of Drosophila melanogaster

The Drosophila photoreceptor potential is thought to be composed of discrete unit potentials called bumps. The steady-state receptor potential and the accompanying voltage fluctuations were recorded intracellularly under steady illumination. The occurrence rate, effective amplitude, and duration of the bumps were deduced by assuming a shot noise model. Over a wide range of light intensity, the duration of bumps remained essentially constant (25-30 ms). Below the saturation intensity for the receptor potential, the bump rate was roughly proportional to the intensity, and the adjustment of bumps to smaller size at higher intensity was mainly responsible for the nonlinear behavior of the receptor potential. The reduction in size of bumps at increasing light intensity was found to be due mainly to the diminishing magnitude of the bump current, and not to some other secondary effects. The bump rate saturated at about 3 x 105-106 events/s.     

Dynamics of diluted attractor neural networks with delays

We present an analysis of the attractors of a deterministic dynamics in formal neural networks characterized by binary threshold units and a nonsymmetric connectivity. It is shown that in these networks a stored pattern or a pattern sequence is represented by a cloud of attractors rather than by a single attractor. Dilution, which we describe by a power-law scaling, and delayed couplings are shown to equip this type of network with a dynamic behaviour that is interesting enough for simplified models of biological motor systems. Received: 27 November 1992/Accepted in revised form: 22 September 1993     

The quantal source of area supralinearity of flash responses in Limulus photoreceptors

The time integrals of the responses of dark-adapted Limulus ventral photoreceptors to flashes exhibit a supralinear dependence on intensity at intermediate intensities. By decomposing the responses into their elementary single-photon components ("bumps"), we are able to calculate the overall quantum efficiency and to display the time courses of the bump amplitude and rate of appearance. Since the time course of the flash response is not slow compared with that of the bump, it was necessary, in order to carry out the decomposition, to develop a new technique for noise analysis of dynamic signals. This new technique should have wide applications. Our main finding is that the supralinearity of the flash responses corresponds to an increase in bump amplitude, with little change in bump duration or quantum efficiency. The time courses of the bump rate and of the change in bump amplitude are peaked and have widths similar to that of the response itself. The peaks of the time courses of the bump rate and amplitude displayed against the starting times of the bumps do not coincide and occur approximately 80 and approximately 40 ms, respectively, before the peak of the response. The time from the start of a bump to its centroid is approximately 70 ms, which means that the time at which the bump centroid reaches its maximum follows the response peak by 30 ms. These results impose constraints on possible mechanisms for the amplitude enhancement.     

Molecular basis of amplification in Drosophila phototransduction: roles for G protein,phospholipase C,and diacylglycerol kinase

In Drosophila photoreceptors, the amplification responsible for generating quantum bumps in response to photoisomerization of single rhodopsin molecules has been thought to be mediated downstream of phospholipase C (PLC), since bump amplitudes were reportedly unaffected in mutants with greatly reduced levels of either G protein or PLC. We now find that quantum bumps in such mutants are reduced approximately 3- to 5-fold but are restored to near wild-type values by mutations in the rdgA gene encoding diacylglycerol kinase (DGK) and also by depleting intracellular ATP. The results demonstrate that amplification requires activation of multiple G protein and PLC molecules, identify DGK as a key enzyme regulating amplification, and implicate diacylglycerol as a messenger of excitation in Drosophila phototransduction.     

Can quantum-bumps in photoreceptors be reconstructed from noise-data?

The method of reconstructing quantum bumps in photoreceptor cells from noise data by making use of shot noise theory is critically reviewed. The application of this method produces results irrespective of whether the conditions for reconstructing bumps by the method are satisfied or not and even irrespective of whether at high stimulus intensities quantum bumps exist or not. We argue that at high intensities the concept of quantum bumps indeed becomes physically meaningless and degenerates to a purely mathematical concept. In order to investigate the meaning of the results of the reconstruction method, we submit it to a test model for which bumps and single channel opening events can be evaluated analytically. By comparing the analytical results of the test model with that of the reconstruction method applied to the test model we find: (1) even at low intensities, the reconstructed bump values deviate from the analytical results by up to an order of magnitude due to the variability of the bumps, (2) at high intensities, the reconstruction method produces single chennel opening events rather than anything like a quantum bump. We also find, however, that there is no continuous transition from a bump at low intensities to a single channel event at high intensities.     

Memory Capacity of Networks with Stochastic Binary Synapses

In standard attractor neural network models, specific patterns of activity are stored in the synaptic matrix, so that they become fixed point attractors of the network dynamics. The storage capacity of such networks has been quantified in two ways: the maximal number of patterns that can be stored, and the stored information measured in bits per synapse. In this paper, we compute both quantities in fully connected networks of N binary neurons with binary synapses, storing patterns with coding level , in the large and sparse coding limits (). We also derive finite-size corrections that accurately reproduce the results of simulations in networks of tens of thousands of neurons. These methods are applied to three different scenarios: (1) the classic Willshaw model, (2) networks with stochastic learning in which patterns are shown only once (one shot learning), (3) networks with stochastic learning in which patterns are shown multiple times. The storage capacities are optimized over network parameters, which allows us to compare the performance of the different models. We show that finite-size effects strongly reduce the capacity, even for networks of realistic sizes. We discuss the implications of these results for memory storage in the hippocampus and cerebral cortex.     

Neural network mechanisms underlying stimulus driven variability reduction

It is well established that the variability of the neural activity across trials, as measured by the Fano factor, is elevated. This fact poses limits on information encoding by the neural activity. However, a series of recent neurophysiological experiments have changed this traditional view. Single cell recordings across a variety of species, brain areas, brain states and stimulus conditions demonstrate a remarkable reduction of the neural variability when an external stimulation is applied and when attention is allocated towards a stimulus within a neuron's receptive field, suggesting an enhancement of information encoding. Using an heterogeneously connected neural network model whose dynamics exhibits multiple attractors, we demonstrate here how this variability reduction can arise from a network effect. In the spontaneous state, we show that the high degree of neural variability is mainly due to fluctuation-driven excursions from attractor to attractor. This occurs when, in the parameter space, the network working point is around the bifurcation allowing multistable attractors. The application of an external excitatory drive by stimulation or attention stabilizes one specific attractor, eliminating in this way the transitions between the different attractors and resulting in a net decrease in neural variability over trials. Importantly, non-responsive neurons also exhibit a reduction of variability. Finally, this reduced variability is found to arise from an increased regularity of the neural spike trains. In conclusion, these results suggest that the variability reduction under stimulation and attention is a property of neural circuits.     

Changes of mind in an attractor network of decision-making

Attractor networks successfully account for psychophysical and neurophysiological data in various decision-making tasks. Especially their ability to model persistent activity, a property of many neurons involved in decision-making, distinguishes them from other approaches. Stable decision attractors are, however, counterintuitive to changes of mind. Here we demonstrate that a biophysically-realistic attractor network with spiking neurons, in its itinerant transients towards the choice attractors, can replicate changes of mind observed recently during a two-alternative random-dot motion (RDM) task. Based on the assumption that the brain continues to evaluate available evidence after the initiation of a decision, the network predicts neural activity during changes of mind and accurately simulates reaction times, performance and percentage of changes dependent on difficulty. Moreover, the model suggests a low decision threshold and high incoming activity that drives the brain region involved in the decision-making process into a dynamical regime close to a bifurcation, which up to now lacked evidence for physiological relevance. Thereby, we further affirmed the general conformance of attractor networks with higher level neural processes and offer experimental predictions to distinguish nonlinear attractor from linear diffusion models.     

The Problem of Colliding Networks and its Relation to Cell Fusion and?Cancer

Cell fusion, a process that merges two or more cells into one, is required for normal development and has been explored as a tool for stem cell therapy. It has also been proposed that cell fusion causes cancer and contributes to its progression. These functions rely on a poorly understood ability of cell fusion to create new cell types. We suggest that this ability can be understood by considering cells as attractor networks whose basic property is to adopt a set of distinct, stable, self-maintaining states called attractors. According to this view, fusion of two cell types is a collision of two networks that have adopted distinct attractors. To learn how these networks reach a consensus, we model cell fusion computationally. To do so, we simulate patterns of gene activities using a formalism developed to simulate patterns of memory in neural networks. We find that the hybrid networks can assume attractors that are unrelated to parental attractors, implying that cell fusion can create new cell types by nearly instantaneously moving cells between attractors. We also show that hybrid networks are prone to assume spurious attractors, which are emergent and sporadic network states. This finding means that cell fusion can produce abnormal cell types, including cancerous types, by placing cells into normally inaccessible spurious states. Finally, we suggest that the problem of colliding networks has general significance in many processes represented by attractor networks, including biological, social, and political phenomena.     

Bump amplitude,but not latency and time-course,can depend on position of illumination in Limulus photoreceptor cells

Bumps, the responses evoked by single photons in the ventral photoreceptor of Limulus polyphemus, were measured under voltage clamp conditions. The bumps were evoked by illuminating the photoreceptor either with a global flash or a small light spot (diameter about 5 m) which covers only 0.25% of the light-sensitive part of the cell membrane. The light energy of both flash types was adjusted so that each flash on average evoked one bump. Parameters of bumps evoked by local light spots in various membrane areas were compared with those evoked by light flashes which illuminated the whole photoreceptor. The results show that the bump amplitude depends on the location of the illumination. Membrane areas were found where the average value of the bump amplitude was either smaller or larger for a spot illumination than for a whole cell illumination. The latency and the shape (e.g. width) of the bumps does not depend on the location of the illumination.