Variable threshold as a model for selective attention, (de)sensitization,and anesthesia in associative neural networks

We study the influence of a variable neuronal threshold on fixed points and convergence rates of an associative neural network in the presence of noise. We allow a random distribution in the activity levels of the patterns stored, and a modification to the standard Hebbian learning rule is proposed for this purpose. There is a threshold at which the retrieval ability, including the average final overlap and the convergence rate, is optimized for patterns with a particular activity level at a given noise level. This type of selective attention to one class of patterns with a certain activity level may be obtained at the cost of reducing the retrieval ability of the network for patterns with different activity levels. The effects of a constant threshold independent of noise, time, and pattern are discussed. For high-(low-) activity patterns, the average final overlap is shown to be increased at high noise levels and decreased at low noise levels by a negative (positive) constant threshold, whereas a positive (negative) threshold always reduces the final average overlap. When the magnitude of the constant threshold exceeds a critical value, there is no retrieval. Rates of convergence towards the stored pattern with negative (positive) thresholds are greater than those with positive (negative) thresholds. These results are related to (de)sensitization and anesthesia. For certain threshold values and patterns with certain activity levels, hysteresis appears in the plot of the average final overlap versus the noise level, even for first order interactions. We make the analogy between the pattern-dependent neuronal threshold proposed in the present paper and the task-related modulation in neuronal excitability determined by cognitive factors, such as the attentional state of a higher animal. A constant threshold is associated with overall changes in neuronal excitability caused, e.g., by various drugs and physical injuries. Neurophysiological evidence of a dynamically variable neuronal threshold, such as accommodation and potentiation, is presented.     

Izhikevich神经元网络的同步与联想记忆

联想记忆是人脑的一项重要功能。以Izhikevich神经元模型为节点,构建神经网络,神经元之间采用全连结的方式;以神经元群体的时空编码（spatio-temporal coding）理论研究所构建神经网络的联想记忆功能。在加入高斯白噪声的情况下,调节网络中神经元之间的连接强度的大小,当连接强度和噪声强度达到一个阈值时网络中部分神经元同步放电,实现了存储模式的联想记忆与恢复。仿真结果表明,神经元之间的连接强度在联想记忆的过程中发挥了重要的作用,噪声可以促使神经元间的同步放电,有助于神经网络实现存储模式的联想记忆与恢复。     

Are binary synapses superior to graded weight representations in stochastic attractor networks?

Synaptic plasticity is an underlying mechanism of learning and memory in neural systems, but it is controversial whether synaptic efficacy is modulated in a graded or binary manner. It has been argued that binary synaptic weights would be less susceptible to noise than graded weights, which has impelled some theoretical neuroscientists to shift from the use of graded to binary weights in their models. We compare retrieval performance of models using both binary and graded weight representations through numerical simulations of stochastic attractor networks. We also investigate stochastic attractor models using multiple discrete levels of weight states, and then investigate the optimal threshold for dilution of binary weight representations. Our results show that a binary weight representation is not less susceptible to noise than a graded weight representation in stochastic attractor models, and we find that the load capacities with an increasing number of weight states rapidly reach the load capacity with graded weights. The optimal threshold for dilution of binary weight representations under stochastic conditions occurs when approximately 50% of the smallest weights are set to zero.     

Stochastic resonance in the speed of memory retrieval

The stochastic resonance (SR) phenomenon in human cognition (memory retrieval speed for arithmetical multiplication rules) is addressed in a behavioral and neurocomputational study. The results of an experiment in which performance was monitored for various magnitudes of acoustic noise are presented. The average response time was found to be minimal for some optimal noise level. Moreover, it was shown that the optimal noise level and the magnitude of the SR effect depend on the difficulty of the task. A computational framework based on leaky accumulators that integrate noisy information and provide the output upon reaching a threshold criterion is used to illustrate the observed phenomena.     

On the dynamic organization of memory

The interaction of memory structures and retrieval dynamics is discussed. A mathematical model for associative free recall is presented to support the view that the organization of simple processing units plays an important role in the retrieval of memory traces. Computer simulations show that "flexibility" and "fidelity" of the dynamics strongly depend on the network structure, the amplification and decay parameters, and the noise term.     

Extremely Dilute Modular Neuronal Networks: Neocortical Memory Retrieval Dynamics

A model of columnar networks of neocortical association areas is studied. The neuronal network is composed of many Hebbian autoassociators, or modules, each of which interacts with a relatively small number of the others, randomly chosen. Any module encodes and stores a number of elementary percepts, or features. Memory items, or patterns, are peculiar combinations of features sparsely distributed over the multi-modular network. Any feature stored in any module can be involved in several of the stored patterns; feature-sharing is in fact source of local ambiguities and, consequently, a potential cause of erroneous memory retrieval spreading through the model network in pattern completion tasks.The memory retrieval dynamics of the large modular autoassociator is investigated by combining mathematical analysis and numerical simulations. An oscillatory retrieval process is proposed that is very efficient in overcoming feature-sharing drawbacks; it requires a mechanism that modulates the robustness of local attractors to noise, and neuronal activity sparseness such that quiescent and active modules are about equally noisy to any post-synaptic module.Moreover, it is shown that statistical correlation between 'kinds' of features across the set of memory patterns can be exploited to obtain a more efficient achievement of memory retrieval capabilities.It is also shown that some spots of the network cannot be reached by retrieval activity spread if they are not directly cued by the stimulus. The locations of these activity isles depend on the pattern to retrieve, while their extension only depends (in large networks) on statistics of inter-modular connections and stored patterns. The existence of activity isles determines an upper-bound to retrieval quality that does not depend on the specific retrieval dynamics adopted, nor on whether feature-sharing is permitted. The oscillatory retrieval process nearly saturates this bound.     

Generic Information Can Retrieve Known Biological Associations: Implications for Biomedical Knowledge Discovery

Motivation

Weighted semantic networks built from text-mined literature can be used to retrieve known protein-protein or gene-disease associations, and have been shown to anticipate associations years before they are explicitly stated in the literature. Our text-mining system recognizes over 640,000 biomedical concepts: some are specific (i.e., names of genes or proteins) others generic (e.g., ‘Homo sapiens’). Generic concepts may play important roles in automated information retrieval, extraction, and inference but may also result in concept overload and confound retrieval and reasoning with low-relevance or even spurious links. Here, we attempted to optimize the retrieval performance for protein-protein interactions (PPI) by filtering generic concepts (node filtering) or links to generic concepts (edge filtering) from a weighted semantic network. First, we defined metrics based on network properties that quantify the specificity of concepts. Then using these metrics, we systematically filtered generic information from the network while monitoring retrieval performance of known protein-protein interactions. We also systematically filtered specific information from the network (inverse filtering), and assessed the retrieval performance of networks composed of generic information alone.

Results

Filtering generic or specific information induced a two-phase response in retrieval performance: initially the effects of filtering were minimal but beyond a critical threshold network performance suddenly drops. Contrary to expectations, networks composed exclusively of generic information demonstrated retrieval performance comparable to unfiltered networks that also contain specific concepts. Furthermore, an analysis using individual generic concepts demonstrated that they can effectively support the retrieval of known protein-protein interactions. For instance the concept “binding” is indicative for PPI retrieval and the concept “mutation abnormality” is indicative for gene-disease associations.

Conclusion

Generic concepts are important for information retrieval and cannot be removed from semantic networks without negative impact on retrieval performance.     

Optimum signal in a simple neuronal model with signal-dependent noise

How does the information about a signal in neural threshold crossings depend on the noise acting upon it? Two models are explored, a binary McCulloch and Pitts (threshold exceedance) model and a model of waiting time to exceedance--a discrete-time version of interspike intervals. If noise grows linearly with the signal, we find the best identification of the signal in terms of the Fisher information is for signals that do not reach the threshold in the absence of noise. Identification attains the same precision under weak and strong signals, but the coding range decreases at both extremes of signal level. We compare the results obtained for Fisher information with those using related first and second moment measures. The maximum obtainable information is plotted as a function of the ratio of noise to signal.     

Effects of Poisson noise in a IF model with STDP and spontaneous replay of periodic spatiotemporal patterns,in absence of cue stimulation

We consider a network of leaky integrate and fire neurons, whose learning mechanism is based on the Spike-Timing-Dependent Plasticity. The spontaneous temporal dynamic of the system is studied, including its storage and replay properties, when a Poissonian noise is added to the post-synaptic potential of the units. The temporal patterns stored in the network are periodic spatiotemporal patterns of spikes. We observe that, even in absence of a cue stimulation, the spontaneous dynamics induced by the noise is a sort of intermittent replay of the patterns stored in the connectivity and a phase transition between a replay and non-replay regime exists at a critical value of the spiking threshold. We characterize this transition by measuring the order parameter and its fluctuations.     

Imaging performance of phase-contrast breast computed tomography with synchrotron radiation and a CdTe photon-counting detector

PurposeWithin the SYRMA-CT collaboration based at the ELETTRA synchrotron radiation (SR) facility the authors investigated the imaging performance of the phase-contrast computed tomography (CT) system dedicated to monochromatic in vivo 3D imaging of the female breast, for breast cancer diagnosis.MethodsTest objects were imaged at 38 keV using monochromatic SR and a high-resolution CdTe photon-counting detector. Signal and noise performance were evaluated using modulation transfer function (MTF) and noise power spectrum. The analysis was performed on the images obtained with the application of a phase retrieval algorithm as well as on those obtained without phase retrieval. The contrast to noise ratio (CNR) and the capability of detecting test microcalcification clusters and soft masses were investigated.ResultsFor a voxel size of (60 μm)³, images without phase retrieval showed higher spatial resolution (6.7 mm⁻¹ at 10% MTF) than corresponding images with phase retrieval (2.5 mm⁻¹). Phase retrieval produced a reduction of the noise level and an increase of the CNR by more than one order of magnitude, compared to raw phase-contrast images. Microcalcifications with a diameter down to 130 μm could be detected in both types of images.ConclusionsThe investigation on test objects indicates that breast CT with a monochromatic SR source is technically feasible in terms of spatial resolution, image noise and contrast, for in vivo 3D imaging with a dose comparable to that of two-view mammography. Images obtained with the phase retrieval algorithm showed the best performance in the trade-off between spatial resolution and image noise.     

Emergence of resonances in neural systems: the interplay between adaptive threshold and short-term synaptic plasticity

In this work we study the detection of weak stimuli by spiking (integrate-and-fire) neurons in the presence of certain level of noisy background neural activity. Our study has focused in the realistic assumption that the synapses in the network present activity-dependent processes, such as short-term synaptic depression and facilitation. Employing mean-field techniques as well as numerical simulations, we found that there are two possible noise levels which optimize signal transmission. This new finding is in contrast with the classical theory of stochastic resonance which is able to predict only one optimal level of noise. We found that the complex interplay between adaptive neuron threshold and activity-dependent synaptic mechanisms is responsible for this new phenomenology. Our main results are confirmed by employing a more realistic FitzHugh-Nagumo neuron model, which displays threshold variability, as well as by considering more realistic stochastic synaptic models and realistic signals such as poissonian spike trains.     

Evolution of robustness to noise and mutation in gene expression dynamics

Phenotype of biological systems needs to be robust against mutation in order to sustain themselves between generations. On the other hand, phenotype of an individual also needs to be robust against fluctuations of both internal and external origins that are encountered during growth and development. Is there a relationship between these two types of robustness, one during a single generation and the other during evolution? Could stochasticity in gene expression have any relevance to the evolution of these types of robustness? Robustness can be defined by the sharpness of the distribution of phenotype; the variance of phenotype distribution due to genetic variation gives a measure of 'genetic robustness', while that of isogenic individuals gives a measure of 'developmental robustness'. Through simulations of a simple stochastic gene expression network that undergoes mutation and selection, we show that in order for the network to acquire both types of robustness, the phenotypic variance induced by mutations must be smaller than that observed in an isogenic population. As the latter originates from noise in gene expression, this signifies that the genetic robustness evolves only when the noise strength in gene expression is larger than some threshold. In such a case, the two variances decrease throughout the evolutionary time course, indicating increase in robustness. The results reveal how noise that cells encounter during growth and development shapes networks' robustness to stochasticity in gene expression, which in turn shapes networks' robustness to mutation. The necessary condition for evolution of robustness, as well as the relationship between genetic and developmental robustness, is derived quantitatively through the variance of phenotypic fluctuations, which are directly measurable experimentally.     

Capacity analysis in multi-state synaptic models: a retrieval probability perspective

We define the memory capacity of networks of binary neurons with finite-state synapses in terms of retrieval probabilities of learned patterns under standard asynchronous dynamics with a predetermined threshold. The threshold is set to control the proportion of non-selective neurons that fire. An optimal inhibition level is chosen to stabilize network behavior. For any local learning rule we provide a computationally efficient and highly accurate approximation to the retrieval probability of a pattern as a function of its age. The method is applied to the sequential models (Fusi and Abbott, Nat Neurosci 10:485–493, 2007) and meta-plasticity models (Fusi et al., Neuron 45(4):599–611, 2005; Leibold and Kempter, Cereb Cortex 18:67–77, 2008). We show that as the number of synaptic states increases, the capacity, as defined here, either plateaus or decreases. In the few cases where multi-state models exceed the capacity of binary synapse models the improvement is small.     

Threshold fatigue and information transfer

Neurons in vivo must process sensory information in the presence of significant noise. It is thus plausible to assume that neural systems have developed mechanisms to reduce this noise. Theoretical studies have shown that threshold fatigue (i.e. cumulative increases in the threshold during repetitive firing) could lead to noise reduction at certain frequencies bands and thus improved signal transmission as well as noise increases and decreased signal transmission at other frequencies: a phenomenon called noise shaping. There is, however, no experimental evidence that threshold fatigue actually occurs and, if so, that it will actually lead to noise shaping. We analyzed action potential threshold variability in intracellular recordings in vivo from pyramidal neurons in weakly electric fish and found experimental evidence for threshold fatigue: an increase in instantaneous firing rate was on average accompanied by an increase in action potential threshold. We show that, with a minor modification, the standard Hodgkin–Huxley model can reproduce this phenomenon. We next compared the performance of models with and without threshold fatigue. Our results show that threshold fatigue will lead to a more regular spike train as well as robustness to intrinsic noise via noise shaping. We finally show that the increased/reduced noise levels due to threshold fatigue correspond to decreased/increased information transmission at different frequencies. Action Editor: David Golomb     

Spiking regularity in a noisy small-world neuronal network

The regularity of spiking oscillations is studied in the networks with different topological structures. The network is composed of coupled Fitz-Hugh-Nagumo neurons driven by colored noise. The investigation illustrates that the spike train in both the regular and the Watts-Strogatz small-world neuronal networks can show the best regularity at a moderate noise intensity, indicating the existence of coherence resonance. Moreover, the temporal coherence of the spike train in the small-world network is superior to that in a regular network due to the increase of the randomness of the network topology. Besides the noise intensity, the spiking regularity can be optimized by tuning the randomness of the network topological structure or by tuning the correlation time of the colored noise. In particular, under the cooperation of the small-world topology and the correlation time, the spike train with good regularity could sustain a large magnitude of the local noise.     

Resonances and noise in a stochastic Hindmarsh-Rose model of thalamic neurons

Thalamic neurons exhibit subthreshold resonance when stimulated with small sine wave signals of varying frequency and stochastic resonance when noise is added to these signals. We study a stochastic Hindmarsh-Rose model using Monte-Carlo simulations to investigate how noise, in conjunction with subthreshold resonance, leads to a preferred frequency in the firing pattern. The resulting stochastic resonance (SR) exhibits a preferred firing frequency that is approximately exponential in its dependence on the noise amplitude. In similar experiments, frequency dependent SR is found in the reliability of detection of alpha-function inputs under noise, which are more realistic inputs for neurons. A mathematical analysis of the equations reveals that the frequency preference arises from the dynamics of the slow variable. Noise can then transfer the resonance over the firing threshold because of the proximity of the fast subsystem to a Hopf bifurcation point. Our results may have implications for the behavior of thalamic neurons in a network, with noise switching the membrane potential between different resonance modes.     

Age-Related Changes in the Functional Network Underlying Specific and General Autobiographical Memory Retrieval: A Pivotal Role for the Anterior Cingulate Cortex

Age-related changes in autobiographical memory (AM) recall are characterized by a decline in episodic details, while semantic aspects are spared. This deleterious effect is supposed to be mediated by an inefficient recruitment of executive processes during AM retrieval. To date, contrasting evidence has been reported on the neural underpinning of this decline, and none of the previous studies has directly compared the episodic and semantic aspects of AM in elderly. We asked 20 young and 17 older participants to recall specific and general autobiographical events (i.e., episodic and semantic AM) elicited by personalized cues while recording their brain activity by means of fMRI. At the behavioral level, we confirmed that the richness of episodic AM retrieval is specifically impoverished in aging and that this decline is related to the reduction of executive functions. At the neural level, in both age groups, we showed the recruitment of a large network during episodic AM retrieval encompassing prefrontal, cortical midline and posterior regions, and medial temporal structures, including the hippocampus. This network was very similar, but less extended, during semantic AM retrieval. Nevertheless, a greater activity was evidenced in the dorsal anterior cingulate cortex (dACC) during episodic, compared to semantic AM retrieval in young participants, and a reversed pattern in the elderly. Moreover, activity in dACC during episodic AM retrieval was correlated with inhibition and richness of memories in both groups. Our findings shed light on the direct link between episodic AM retrieval, executive control, and their decline in aging, proposing a possible neuronal signature. They also suggest that increased activity in dACC during semantic AM retrieval in the elderly could be seen as a compensatory mechanism underpinning successful AM performance observed in aging. These results are discussed in the framework of recently proposed models of neural reorganization in aging.