Balance between Noise and Information Flow Maximizes Set Complexity of Network Dynamics

Boolean networks have been used as a discrete model for several biological systems, including metabolic and genetic regulatory networks. Due to their simplicity they offer a firm foundation for generic studies of physical systems. In this work we show, using a measure of context-dependent information, set complexity, that prior to reaching an attractor, random Boolean networks pass through a transient state characterized by high complexity. We justify this finding with a use of another measure of complexity, namely, the statistical complexity. We show that the networks can be tuned to the regime of maximal complexity by adding a suitable amount of noise to the deterministic Boolean dynamics. In fact, we show that for networks with Poisson degree distributions, all networks ranging from subcritical to slightly supercritical can be tuned with noise to reach maximal set complexity in their dynamics. For networks with a fixed number of inputs this is true for near-to-critical networks. This increase in complexity is obtained at the expense of disruption in information flow. For a large ensemble of networks showing maximal complexity, there exists a balance between noise and contracting dynamics in the state space. In networks that are close to critical the intrinsic noise required for the tuning is smaller and thus also has the smallest effect in terms of the information processing in the system. Our results suggest that the maximization of complexity near to the state transition might be a more general phenomenon in physical systems, and that noise present in a system may in fact be useful in retaining the system in a state with high information content.     

Ensembles of Spiking Neurons with Noise Support Optimal Probabilistic Inference in a Dynamically Changing Environment

It has recently been shown that networks of spiking neurons with noise can emulate simple forms of probabilistic inference through “neural sampling”, i.e., by treating spikes as samples from a probability distribution of network states that is encoded in the network. Deficiencies of the existing model are its reliance on single neurons for sampling from each random variable, and the resulting limitation in representing quickly varying probabilistic information. We show that both deficiencies can be overcome by moving to a biologically more realistic encoding of each salient random variable through the stochastic firing activity of an ensemble of neurons. The resulting model demonstrates that networks of spiking neurons with noise can easily track and carry out basic computational operations on rapidly varying probability distributions, such as the odds of getting rewarded for a specific behavior. We demonstrate the viability of this new approach towards neural coding and computation, which makes use of the inherent parallelism of generic neural circuits, by showing that this model can explain experimentally observed firing activity of cortical neurons for a variety of tasks that require rapid temporal integration of sensory information.     

Signal propagation in feedforward neuronal networks with unreliable synapses

In this paper, we systematically investigate both the synfire propagation and firing rate propagation in feedforward neuronal network coupled in an all-to-all fashion. In contrast to most earlier work, where only reliable synaptic connections are considered, we mainly examine the effects of unreliable synapses on both types of neural activity propagation in this work. We first study networks composed of purely excitatory neurons. Our results show that both the successful transmission probability and excitatory synaptic strength largely influence the propagation of these two types of neural activities, and better tuning of these synaptic parameters makes the considered network support stable signal propagation. It is also found that noise has significant but different impacts on these two types of propagation. The additive Gaussian white noise has the tendency to reduce the precision of the synfire activity, whereas noise with appropriate intensity can enhance the performance of firing rate propagation. Further simulations indicate that the propagation dynamics of the considered neuronal network is not simply determined by the average amount of received neurotransmitter for each neuron in a time instant, but also largely influenced by the stochastic effect of neurotransmitter release. Second, we compare our results with those obtained in corresponding feedforward neuronal networks connected with reliable synapses but in a random coupling fashion. We confirm that some differences can be observed in these two different feedforward neuronal network models. Finally, we study the signal propagation in feedforward neuronal networks consisting of both excitatory and inhibitory neurons, and demonstrate that inhibition also plays an important role in signal propagation in the considered networks.     

Diversity of temporal correlations between genes in models of noisy and noiseless gene networks

Gene regulatory networks (GRNs) are parallel information processing systems, binding past events to future actions. Since cell types stably remain in restricted subsets of the possible states of the GRN, they are likely the dynamical attractors of the GRN. These attractors differ in which genes are active and in the amount of information propagating within the network. Using mutual information (I) as a measure of information propagation between genes in a GRN, modeled as finite-sized Random Boolean Networks (RBN), we study how the dynamical regime of the GRN affects I within attractors (I(A)). The spectra of I(A) of individual RBNs are found to be scattered and diverse, and distributions of I(A) of ensembles are non-trivial and change shape with mean connectivity. Mean and diversity of I(A) values maximize in the chaotic near-critical regime, whereas ordered near-critical networks are the best at retaining the distinctiveness of each attractor's I(A) with noise. The results suggest that selection likely favors near-critical GRNs as these both maximize mean and diversity of I(A), and are the most robust to noise. We find similar I(A) distributions in delayed stochastic models of GRNs. For a particular stochastic GRN, we show that both mean and variance of I(A) have local maxima as its connectivity and noise levels are varied, suggesting that the conclusions for the Boolean network models may be generalizable to more realistic models of GRNs.     

Coding of time-dependent stimuli in homogeneous and heterogeneous neural populations

We compare the information transmission of a time-dependent signal by two types of uncoupled neuron populations that differ in their sources of variability: i) a homogeneous population whose units receive independent noise and ii) a deterministic heterogeneous population, where each unit exhibits a different baseline firing rate (’disorder’). Our criterion for making both sources of variability quantitatively comparable is that the interspike-interval distributions are identical for both systems. Numerical simulations using leaky integrate-and-fire neurons unveil that a non-zero amount of both noise or disorder maximizes the encoding efficiency of the homogeneous and heterogeneous system, respectively, as a particular case of suprathreshold stochastic resonance. Our findings thus illustrate that heterogeneity can render similarly profitable effects for neuronal populations as dynamic noise. The optimal noise/disorder depends on the system size and the properties of the stimulus such as its intensity or cutoff frequency. We find that weak stimuli are better encoded by a noiseless heterogeneous population, whereas for strong stimuli a homogeneous population outperforms an equivalent heterogeneous system up to a moderate noise level. Furthermore, we derive analytical expressions of the coherence function for the cases of very strong noise and of vanishing intrinsic noise or heterogeneity, which predict the existence of an optimal noise intensity. Our results show that, depending on the type of signal, noise as well as heterogeneity can enhance the encoding performance of neuronal populations.     

Activated cortical states: Experiments, analyses and models

In awake animals, the cerebral cortex displays an "activated" state, with distinct characteristics compared to other states like slow-wave sleep or anesthesia. These characteristics include a sustained depolarized membrane potential (V(m)) and irregular firing activity. In the present paper, we evaluate our understanding of cortical activated states from a computational neuroscience point of view. We start by reviewing the electrophysiological characteristics of activated cortical states based on recordings and analysis performed in awake cat association cortex. These analyses show that cortical activity is characterized by an apparent Poisson-distributed stochastic dynamics, both at the single-cell and population levels, and that single cells display a high-conductance state dominated by inhibition. We next overview computational models of the "awake" cortex, and perform the same analyses as in the experiments. Many properties identified experimentally are indeed reproduced by models, such as depolarized V(m), irregular firing with apparent Poisson statistics, and the determinant role of inhibitory fluctuations on spiking. However, other features are not well reproduced, such as firing statistics and the conductance state of the membrane, suggesting that the network state displayed by models is not entirely correct. We also show how networks can approach a correct conductance state, suggesting ways by which future models will generate activity fully consistent with experimental data.     

An analysis of the reliability phenomenon in the FitzHugh-Nagumo model

The reliability of single neurons on realistic stimuli has been experimentally confirmed in a wide variety of animal preparations. We present a theoretical study of the reliability phenomenon in the FitzHugh-Nagumo model on white Gaussian stimulation. The analysis of the model's dynamics is performed in three regimes—the excitable, bistable, and oscillatory ones. We use tools from the random dynamical systems theory, such as the pullbacks and the estimation of the Lyapunov exponents and rotation number. The results show that for most stimulus intensities, trajectories converge to a single stochastic equilibrium point, and the leading Lyapunov exponent is negative. Consequently, in these regimes the discharge times are reliable in the sense that repeated presentation of the same aperiodic input segment evokes similar firing times after some transient time. Surprisingly, for a certain range of stimulus intensities, unreliable firing is observed due to the onset of stochastic chaos, as indicated by the estimated positive leading Lyapunov exponents. For this range of stimulus intensities, stochastic chaos occurs in the bistable regime and also expands in adjacent parts of the excitable and oscillating regimes. The obtained results are valuable in the explanation of experimental observations concerning the reliability of neurons stimulated with broad-band Gaussian inputs. They reveal two distinct neuronal response types. In the regime where the first Lyapunov has negative values, such inputs eventually lead neurons to reliable firing, and this suggests that any observed variance of firing times in reliability experiments is mainly due to internal noise. In the regime with positive Lyapunov exponents, the source of unreliable firing is stochastic chaos, a novel phenomenon in the reliability literature, whose origin and function need further investigation.     

Effect of the topology and delayed interactions in neuronal networks synchronization

As important as the intrinsic properties of an individual nervous cell stands the network of neurons in which it is embedded and by virtue of which it acquires great part of its responsiveness and functionality. In this study we have explored how the topological properties and conduction delays of several classes of neural networks affect the capacity of their constituent cells to establish well-defined temporal relations among firing of their action potentials. This ability of a population of neurons to produce and maintain a millisecond-precise coordinated firing (either evoked by external stimuli or internally generated) is central to neural codes exploiting precise spike timing for the representation and communication of information. Our results, based on extensive simulations of conductance-based type of neurons in an oscillatory regime, indicate that only certain topologies of networks allow for a coordinated firing at a local and long-range scale simultaneously. Besides network architecture, axonal conduction delays are also observed to be another important factor in the generation of coherent spiking. We report that such communication latencies not only set the phase difference between the oscillatory activity of remote neural populations but determine whether the interconnected cells can set in any coherent firing at all. In this context, we have also investigated how the balance between the network synchronizing effects and the dispersive drift caused by inhomogeneities in natural firing frequencies across neurons is resolved. Finally, we show that the observed roles of conduction delays and frequency dispersion are not particular to canonical networks but experimentally measured anatomical networks such as the macaque cortical network can display the same type of behavior.     

State-Dependent Effects of Na Channel Noise on Neuronal Burst Generation

We explore the effects of stochastic sodium (Na) channel activation on the variability and dynamics of spiking and bursting in a model neuron. The complete model segregates Hodgin-Huxley-type currents into two compartments, and undergoes applied current-dependent bifurcations between regimes of periodic bursting, chaotic bursting, and tonic spiking. Noise is added to simulate variable, finite sizes of the population of Na channels in the fast spiking compartment.During tonic firing, Na channel noise causes variability in interspike intervals (ISIs). The variance, as well as the sensitivity to noise, depend on the model's biophysical complexity. They are smallest in an isolated spiking compartment; increase significantly upon coupling to a passive compartment; and increase again when the second compartment also includes slow-acting currents. In this full model, sufficient noise can convert tonic firing into bursting.During bursting, the actions of Na channel noise are state-dependent. The higher the noise level, the greater the jitter in spike timing within bursts. The noise makes the burst durations of periodic regimes variable, while decreasing burst length duration and variance in a chaotic regime. Na channel noise blurs the sharp transitions of spike time and burst length seen at the bifurcations of the noise-free model. Close to such a bifurcation, the burst behaviors of previously periodic and chaotic regimes become essentially indistinguishable.We discuss biophysical mechanisms, dynamical interpretations and physiological implications. We suggest that noise associated with finite populations of Na channels could evoke very different effects on the intrinsic variability of spiking and bursting discharges, depending on a biological neuron's complexity and applied current-dependent state. We find that simulated channel noise in the model neuron qualitatively replicates the observed variability in burst length and interburst interval in an isolated biological bursting neuron.     

Spike-based population coding and working memory

Compelling behavioral evidence suggests that humans can make optimal decisions despite the uncertainty inherent in perceptual or motor tasks. A key question in neuroscience is how populations of spiking neurons can implement such probabilistic computations. In this article, we develop a comprehensive framework for optimal, spike-based sensory integration and working memory in a dynamic environment. We propose that probability distributions are inferred spike-per-spike in recurrently connected networks of integrate-and-fire neurons. As a result, these networks can combine sensory cues optimally, track the state of a time-varying stimulus and memorize accumulated evidence over periods much longer than the time constant of single neurons. Importantly, we propose that population responses and persistent working memory states represent entire probability distributions and not only single stimulus values. These memories are reflected by sustained, asynchronous patterns of activity which make relevant information available to downstream neurons within their short time window of integration. Model neurons act as predictive encoders, only firing spikes which account for new information that has not yet been signaled. Thus, spike times signal deterministically a prediction error, contrary to rate codes in which spike times are considered to be random samples of an underlying firing rate. As a consequence of this coding scheme, a multitude of spike patterns can reliably encode the same information. This results in weakly correlated, Poisson-like spike trains that are sensitive to initial conditions but robust to even high levels of external neural noise. This spike train variability reproduces the one observed in cortical sensory spike trains, but cannot be equated to noise. On the contrary, it is a consequence of optimal spike-based inference. In contrast, we show that rate-based models perform poorly when implemented with stochastically spiking neurons.     

Increased Firing Irregularity as an Emergent Property of Neural-State Transition in Monkey Prefrontal Cortex

Flexible behaviors are organized by complex neural networks in the prefrontal cortex. Recent studies have suggested that such networks exhibit multiple dynamical states, and can switch rapidly from one state to another. In many complex systems such as the brain, the early-warning signals that may predict whether a critical threshold for state transitions is approaching are extremely difficult to detect. We hypothesized that increases in firing irregularity are a crucial measure for predicting state transitions in the underlying neuronal circuits of the prefrontal cortex. We used both experimental and theoretical approaches to test this hypothesis. Experimentally, we analyzed activities of neurons in the prefrontal cortex while monkeys performed a maze task that required them to perform actions to reach a goal. We observed increased firing irregularity before the activity changed to encode goal-to-action information. Theoretically, we constructed theoretical generic neural networks and demonstrated that changes in neuronal gain on functional connectivity resulted in a loss of stability and an altered state of the networks, accompanied by increased firing irregularity. These results suggest that assessing the temporal pattern of neuronal fluctuations provides important clues regarding the state stability of the prefrontal network. We also introduce a novel scheme that the prefrontal cortex functions in a metastable state near the critical point of bifurcation. According to this scheme, firing irregularity in the prefrontal cortex indicates that the system is about to change its state and the flow of information in a flexible manner, which is essential for executive functions. This metastable and/or critical dynamical state of the prefrontal cortex may account for distractibility and loss of flexibility in the prefrontal cortex in major mental illnesses such as schizophrenia.     

Generation of synaptic noise: selective involvement of neuronal subsets.

All central neurons are subjected to continuous and random variations of their membrane potential because of "spontaneous" activity in their presynaptic afferents. This activity, which is called synaptic noise, is presumed to be responsible for the uncertainty of the input-output relation in these cells. In the Mauthner cell of teleosts, noise is mainly inhibitory, and is generated by the release of neurotransmitter in a probabilistic manner. This inhibitory activity has been studied in detail previously. Taking advantage of this understanding, we have constructed a model of the inhibitory networks and their target in order to determine the conditions required to reproduce the main stochastic aspects of synaptic noise. We have used a combination of computer simulations and simple semianalytical arguments. We conclude that, surprisingly, cells in the presynaptic networks do not contribute equally to these background fluctuations. Rather, noise is generated primarily by the operation of subsets of afferent cells: the spectrum is either dominated by signals originating from interneurons which make few terminals on the Mauthner cell, or by the output of "burster" cells firing spike trains rather than single spikes. Both possibilities lead to specific predictions, one of which has already been verified.     

Bistable,Irregular Firing and Population Oscillations in a Modular Attractor Memory Network

Attractor neural networks are thought to underlie working memory functions in the cerebral cortex. Several such models have been proposed that successfully reproduce firing properties of neurons recorded from monkeys performing working memory tasks. However, the regular temporal structure of spike trains in these models is often incompatible with experimental data. Here, we show that the in vivo observations of bistable activity with irregular firing at the single cell level can be achieved in a large-scale network model with a modular structure in terms of several connected hypercolumns. Despite high irregularity of individual spike trains, the model shows population oscillations in the beta and gamma band in ground and active states, respectively. Irregular firing typically emerges in a high-conductance regime of balanced excitation and inhibition. Population oscillations can produce such a regime, but in previous models only a non-coding ground state was oscillatory. Due to the modular structure of our network, the oscillatory and irregular firing was maintained also in the active state without fine-tuning. Our model provides a novel mechanistic view of how irregular firing emerges in cortical populations as they go from beta to gamma oscillations during memory retrieval.     

Comparison of Coding Capabilities of Type I and Type II Neurons

We consider the dependence of information transfer by neurons on the Type I vs. Type II classification of their dynamics. Our computational study is based on Type I and II implementations of the Morris-Lecar model. It mainly concerns neurons, such as those in the auditory or electrosensory system, which encode band-limited amplitude modulations of a periodic carrier signal, and which fire at random cycles yet preferred phases of this carrier. We first show that the Morris-Lecar model with additive broadband noise ("synaptic noise") can exhibit such firing patterns with either Type I or II dynamics, with or without amplitude modulations of the carrier. We then compare the encoding of band-limited random amplitude modulations for both dynamical types. The comparison relies on a parameter calibration that closely matches firing rates for both models across a range of parameters. In the absence of synaptic noise, Type I performs slightly better than Type II, and its performance is optimal for perithreshold signals. However, Type II performs well over a slightly larger range of inputs, and this range lies mostly in the subthreshold region. Further, Type II performs marginally better than Type I when synaptic noise, which yields more realistic baseline firing patterns, is present in both models. These results are discussed in terms of the tuning and phase locking properties of the models with deterministic and stochastic inputs.     

Membrane Resonance and Stochastic Resonance Modulate Firing Patterns of Thalamocortical Neurons

We examined the interactions of subthreshold membrane resonance and stochastic resonance using whole-cell patch clamp recordings in thalamocortical neurons of rat brain slices, as well as with a Hodgkin-Huxley-type mathematical model of thalamocortical neurons. The neurons exhibited the subthreshold resonance when stimulated with small amplitude sine wave currents of varying frequency, and stochastic resonance when noise was added to sine wave inputs. Stochastic resonance was manifest as a maximum in signal-to-noise ratio of output response to subthreshold periodic input combined with noise. Stochastic resonance in conjunction with subthreshold resonance resulted in action potential patterns that showed frequency selectivity for periodic inputs. Stochastic resonance was maximal near subthreshold resonance frequency and a high noise level was required for detection of high frequency signals. We speculate that combined membrane and stochastic resonances have physiological utility in coupling synaptic activity to preferred firing frequency and in network synchronization under noise.     

Spiking behavior and epileptiform oscillations in a discrete model of cortical neural networks

Summary We investigate the phenomenon of epileptiform activity using a discrete model of cortical neural networks. Our model is reduced to the elementary features of neurons and assumes simplified dynamics of action potentials and postsynaptic potentials. The discrete model provides a comparably high simulation speed which allows the rendering of phase diagrams and simulations of large neural networks in reasonable time. Further the reduction to the basic features of neurons provides insight into the essentials of a possible mechanism of epilepsy. Our computer simulations suggest that the detailed dynamics of postsynaptic and action potentials are not indispensable for obtaining epileptiform behavior on the system level. The simulation results of autonomously evolving networks exhibit a regime in which the network dynamics spontaneously switch between fluctuating and oscillating behavior and produce isolated network spikes without external stimulation. Inhibitory neurons have been found to play an important part in the synchronization of neural firing: an increased number of synapses established by inhibitory neurons onto other neurons induces a transition to the spiking regime. A decreased frequency accompanying the hypersynchronous population activity has only occurred with slow inhibitory postsynaptic potentials.     

Optimal Balance of the Striatal Medium Spiny Neuron Network

Slowly varying activity in the striatum, the main Basal Ganglia input structure, is important for the learning and execution of movement sequences. Striatal medium spiny neurons (MSNs) form cell assemblies whose population firing rates vary coherently on slow behaviourally relevant timescales. It has been shown that such activity emerges in a model of a local MSN network but only at realistic connectivities of and only when MSN generated inhibitory post-synaptic potentials (IPSPs) are realistically sized. Here we suggest a reason for this. We investigate how MSN network generated population activity interacts with temporally varying cortical driving activity, as would occur in a behavioural task. We find that at unrealistically high connectivity a stable winners-take-all type regime is found where network activity separates into fixed stimulus dependent regularly firing and quiescent components. In this regime only a small number of population firing rate components interact with cortical stimulus variations. Around connectivity a transition to a more dynamically active regime occurs where all cells constantly switch between activity and quiescence. In this low connectivity regime, MSN population components wander randomly and here too are independent of variations in cortical driving. Only in the transition regime do weak changes in cortical driving interact with many population components so that sequential cell assemblies are reproducibly activated for many hundreds of milliseconds after stimulus onset and peri-stimulus time histograms display strong stimulus and temporal specificity. We show that, remarkably, this activity is maximized at striatally realistic connectivities and IPSP sizes. Thus, we suggest the local MSN network has optimal characteristics – it is neither too stable to respond in a dynamically complex temporally extended way to cortical variations, nor is it too unstable to respond in a consistent repeatable way. Rather, it is optimized to generate stimulus dependent activity patterns for long periods after variations in cortical excitation.     

Resonances and noise in a stochastic Hindmarsh-Rose model of thalamic neurons

Thalamic neurons exhibit subthreshold resonance when stimulated with small sine wave signals of varying frequency and stochastic resonance when noise is added to these signals. We study a stochastic Hindmarsh-Rose model using Monte-Carlo simulations to investigate how noise, in conjunction with subthreshold resonance, leads to a preferred frequency in the firing pattern. The resulting stochastic resonance (SR) exhibits a preferred firing frequency that is approximately exponential in its dependence on the noise amplitude. In similar experiments, frequency dependent SR is found in the reliability of detection of alpha-function inputs under noise, which are more realistic inputs for neurons. A mathematical analysis of the equations reveals that the frequency preference arises from the dynamics of the slow variable. Noise can then transfer the resonance over the firing threshold because of the proximity of the fast subsystem to a Hopf bifurcation point. Our results may have implications for the behavior of thalamic neurons in a network, with noise switching the membrane potential between different resonance modes.     

Optimal Performance of the Tryptophan Operon of E. coli: A stochastic,Dynamical, Mathematical-Modeling Approach

In this work, we develop a detailed, stochastic, dynamical model for the tryptophan operon of E. coli, and estimate all of the model parameters from reported experimental data. We further employ the model to study the system performance, considering the amount of biochemical noise in the trp level, the system rise time after a nutritional shift, and the amount of repressor molecules necessary to maintain an adequate level of repression, as indicators of the system performance regime. We demonstrate that the level of cooperativity between repressor molecules bound to the first two operators in the trp promoter affects all of the above enlisted performance characteristics. Moreover, the cooperativity level found in the wild-type bacterial strain optimizes a cost-benefit function involving low biochemical noise in the tryptophan level, short rise time after a nutritional shift, and low number of regulatory molecules.     

Population rate coding in recurrent neuronal networks with unreliable synapses

Neuron transmits spikes to postsynaptic neurons through synapses. Experimental observations indicated that the communication between neurons is unreliable. However most modelling and computational studies considered deterministic synaptic interaction model. In this paper, we investigate the population rate coding in an all-to-all coupled recurrent neuronal network consisting of both excitatory and inhibitory neurons connected with unreliable synapses. We use a stochastic on-off process to model the unreliable synaptic transmission. We find that synapses with suitable successful transmission probability can enhance the encoding performance in the case of weak noise; while in the case of strong noise, the synaptic interactions reduce the encoding performance. We also show that several important synaptic parameters, such as the excitatory synaptic strength, the relative strength of inhibitory and excitatory synapses, as well as the synaptic time constant, have significant effects on the performance of the population rate coding. Further simulations indicate that the encoding dynamics of our considered network cannot be simply determined by the average amount of received neurotransmitter for each neuron in a time instant. Moreover, we compare our results with those obtained in the corresponding random neuronal networks. Our numerical results demonstrate that the network randomness has the similar qualitative effect as the synaptic unreliability but not completely equivalent in quantity.