Finite-size and correlation-induced effects in mean-field dynamics

The brain’s activity is characterized by the interaction of a very large number of neurons that are strongly affected by noise. However, signals often arise at macroscopic scales integrating the effect of many neurons into a reliable pattern of activity. In order to study such large neuronal assemblies, one is often led to derive mean-field limits summarizing the effect of the interaction of a large number of neurons into an effective signal. Classical mean-field approaches consider the evolution of a deterministic variable, the mean activity, thus neglecting the stochastic nature of neural behavior. In this article, we build upon two recent approaches that include correlations and higher order moments in mean-field equations, and study how these stochastic effects influence the solutions of the mean-field equations, both in the limit of an infinite number of neurons and for large yet finite networks. We introduce a new model, the infinite model, which arises from both equations by a rescaling of the variables and, which is invertible for finite-size networks, and hence, provides equivalent equations to those previously derived models. The study of this model allows us to understand qualitative behavior of such large-scale networks. We show that, though the solutions of the deterministic mean-field equation constitute uncorrelated solutions of the new mean-field equations, the stability properties of limit cycles are modified by the presence of correlations, and additional non-trivial behaviors including periodic orbits appear when there were none in the mean field. The origin of all these behaviors is then explored in finite-size networks where interesting mesoscopic scale effects appear. This study leads us to show that the infinite-size system appears as a singular limit of the network equations, and for any finite network, the system will differ from the infinite system.     

Memory Capacity of Networks with Stochastic Binary Synapses

In standard attractor neural network models, specific patterns of activity are stored in the synaptic matrix, so that they become fixed point attractors of the network dynamics. The storage capacity of such networks has been quantified in two ways: the maximal number of patterns that can be stored, and the stored information measured in bits per synapse. In this paper, we compute both quantities in fully connected networks of N binary neurons with binary synapses, storing patterns with coding level , in the large and sparse coding limits (). We also derive finite-size corrections that accurately reproduce the results of simulations in networks of tens of thousands of neurons. These methods are applied to three different scenarios: (1) the classic Willshaw model, (2) networks with stochastic learning in which patterns are shown only once (one shot learning), (3) networks with stochastic learning in which patterns are shown multiple times. The storage capacities are optimized over network parameters, which allows us to compare the performance of the different models. We show that finite-size effects strongly reduce the capacity, even for networks of realistic sizes. We discuss the implications of these results for memory storage in the hippocampus and cerebral cortex.     

Understanding evolutionary and ecological dynamics using a continuum limit

Continuum limits in the form of stochastic differential equations are typically used in theoretical population genetics to account for genetic drift or more generally, inherent randomness of the model. In evolutionary game theory and theoretical ecology, however, this method is used less frequently to study demographic stochasticity. Here, we review the use of continuum limits in ecology and evolution. Starting with an individual‐based model, we derive a large population size limit, a (stochastic) differential equation which is called continuum limit. By example of the Wright–Fisher diffusion, we outline how to compute the stationary distribution, the fixation probability of a certain type, and the mean extinction time using the continuum limit. In the context of the logistic growth equation, we approximate the quasi‐stationary distribution in a finite population.     

Representing the UK's cattle herd as static and dynamic networks

Network models are increasingly being used to understand the spread of diseases through sparsely connected populations, with particular interest in the impact of animal movements upon the dynamics of infectious diseases. Detailed data collected by the UK government on the movement of cattle may be represented as a network, where animal holdings are nodes, and an edge is drawn between nodes where a movement of animals has occurred. These network representations may vary from a simple static representation, to a more complex, fully dynamic one where daily movements are explicitly captured. Using stochastic disease simulations, a wide range of network representations of the UK cattle herd are compared. We find that the simpler static network representations are often deficient when compared with a fully dynamic representation, and should therefore be used only with caution in epidemiological modelling. In particular, due to temporal structures within the dynamic network, static networks consistently fail to capture the predicted epidemic behaviour associated with dynamic networks even when parameterized to match early growth rates.     

The Evolutionary Dynamics of Protein-Protein Interaction Networks Inferred from the Reconstruction of Ancient Networks

Cellular functions are based on the complex interplay of proteins, therefore the structure and dynamics of these protein-protein interaction (PPI) networks are the key to the functional understanding of cells. In the last years, large-scale PPI networks of several model organisms were investigated. A number of theoretical models have been developed to explain both the network formation and the current structure. Favored are models based on duplication and divergence of genes, as they most closely represent the biological foundation of network evolution. However, studies are often based on simulated instead of empirical data or they cover only single organisms. Methodological improvements now allow the analysis of PPI networks of multiple organisms simultaneously as well as the direct modeling of ancestral networks. This provides the opportunity to challenge existing assumptions on network evolution. We utilized present-day PPI networks from integrated datasets of seven model organisms and developed a theoretical and bioinformatic framework for studying the evolutionary dynamics of PPI networks. A novel filtering approach using percolation analysis was developed to remove low confidence interactions based on topological constraints. We then reconstructed the ancient PPI networks of different ancestors, for which the ancestral proteomes, as well as the ancestral interactions, were inferred. Ancestral proteins were reconstructed using orthologous groups on different evolutionary levels. A stochastic approach, using the duplication-divergence model, was developed for estimating the probabilities of ancient interactions from today''s PPI networks. The growth rates for nodes, edges, sizes and modularities of the networks indicate multiplicative growth and are consistent with the results from independent static analysis. Our results support the duplication-divergence model of evolution and indicate fractality and multiplicative growth as general properties of the PPI network structure and dynamics.     

Divisive Gain Modulation with Dynamic Stimuli in Integrate-and-Fire Neurons

The modulation of the sensitivity, or gain, of neural responses to input is an important component of neural computation. It has been shown that divisive gain modulation of neural responses can result from a stochastic shunting from balanced (mixed excitation and inhibition) background activity. This gain control scheme was developed and explored with static inputs, where the membrane and spike train statistics were stationary in time. However, input statistics, such as the firing rates of pre-synaptic neurons, are often dynamic, varying on timescales comparable to typical membrane time constants. Using a population density approach for integrate-and-fire neurons with dynamic and temporally rich inputs, we find that the same fluctuation-induced divisive gain modulation is operative for dynamic inputs driving nonequilibrium responses. Moreover, the degree of divisive scaling of the dynamic response is quantitatively the same as the steady-state responses—thus, gain modulation via balanced conductance fluctuations generalizes in a straight-forward way to a dynamic setting.     

Unraveling spurious properties of interaction networks with tailored random networks

We investigate interaction networks that we derive from multivariate time series with methods frequently employed in diverse scientific fields such as biology, quantitative finance, physics, earth and climate sciences, and the neurosciences. Mimicking experimental situations, we generate time series with finite length and varying frequency content but from independent stochastic processes. Using the correlation coefficient and the maximum cross-correlation, we estimate interdependencies between these time series. With clustering coefficient and average shortest path length, we observe unweighted interaction networks, derived via thresholding the values of interdependence, to possess non-trivial topologies as compared to Erd?s-Rényi networks, which would indicate small-world characteristics. These topologies reflect the mostly unavoidable finiteness of the data, which limits the reliability of typically used estimators of signal interdependence. We propose random networks that are tailored to the way interaction networks are derived from empirical data. Through an exemplary investigation of multichannel electroencephalographic recordings of epileptic seizures--known for their complex spatial and temporal dynamics--we show that such random networks help to distinguish network properties of interdependence structures related to seizure dynamics from those spuriously induced by the applied methods of analysis.     

Structure-Function Discrepancy: Inhomogeneity and Delays in Synchronized Neural Networks

The discrepancy between structural and functional connectivity in neural systems forms the challenge in understanding general brain functioning. To pinpoint a mapping between structure and function, we investigated the effects of (in)homogeneity in coupling structure and delays on synchronization behavior in networks of oscillatory neural masses by deriving the phase dynamics of these generic networks. For homogeneous delays, the structural coupling matrix is largely preserved in the coupling between phases, resulting in clustered stationary phase distributions. Accordingly, we found only a small number of synchronized groups in the network. Distributed delays, by contrast, introduce inhomogeneity in the phase coupling so that clustered stationary phase distributions no longer exist. The effect of distributed delays mimicked that of structural inhomogeneity. Hence, we argue that phase (de-)synchronization patterns caused by inhomogeneous coupling cannot be distinguished from those caused by distributed delays, at least not by the naked eye. The here-derived analytical expression for the effective coupling between phases as a function of structural coupling constitutes a direct relationship between structural and functional connectivity. Structural connectivity constrains synchronizability that may be modified by the delay distribution. This explains why structural and functional connectivity bear much resemblance albeit not a one-to-one correspondence. We illustrate this in the context of resting-state activity, using the anatomical connectivity structure reported by Hagmann and others.     

Emergent oscillations in networks of stochastic spiking neurons

Networks of neurons produce diverse patterns of oscillations, arising from the network's global properties, the propensity of individual neurons to oscillate, or a mixture of the two. Here we describe noisy limit cycles and quasi-cycles, two related mechanisms underlying emergent oscillations in neuronal networks whose individual components, stochastic spiking neurons, do not themselves oscillate. Both mechanisms are shown to produce gamma band oscillations at the population level while individual neurons fire at a rate much lower than the population frequency. Spike trains in a network undergoing noisy limit cycles display a preferred period which is not found in the case of quasi-cycles, due to the even faster decay of phase information in quasi-cycles. These oscillations persist in sparsely connected networks, and variation of the network's connectivity results in variation of the oscillation frequency. A network of such neurons behaves as a stochastic perturbation of the deterministic Wilson-Cowan equations, and the network undergoes noisy limit cycles or quasi-cycles depending on whether these have limit cycles or a weakly stable focus. These mechanisms provide a new perspective on the emergence of rhythmic firing in neural networks, showing the coexistence of population-level oscillations with very irregular individual spike trains in a simple and general framework.     

The spectrum of covariance matrices of randomly connected recurrent neuronal networks with linear dynamics

A key question in theoretical neuroscience is the relation between the connectivity structure and the collective dynamics of a network of neurons. Here we study the connectivity-dynamics relation as reflected in the distribution of eigenvalues of the covariance matrix of the dynamic fluctuations of the neuronal activities, which is closely related to the network dynamics’ Principal Component Analysis (PCA) and the associated effective dimensionality. We consider the spontaneous fluctuations around a steady state in a randomly connected recurrent network of stochastic neurons. An exact analytical expression for the covariance eigenvalue distribution in the large-network limit can be obtained using results from random matrices. The distribution has a finitely supported smooth bulk spectrum and exhibits an approximate power-law tail for coupling matrices near the critical edge. We generalize the results to include second-order connectivity motifs and discuss extensions to excitatory-inhibitory networks. The theoretical results are compared with those from finite-size networks and the effects of temporal and spatial sampling are studied. Preliminary application to whole-brain imaging data is presented. Using simple connectivity models, our work provides theoretical predictions for the covariance spectrum, a fundamental property of recurrent neuronal dynamics, that can be compared with experimental data.     

Elements for a general memory structure: properties of recurrent neural networks used to form situation models

We study how individual memory items are stored assuming that situations given in the environment can be represented in the form of synaptic-like couplings in recurrent neural networks. Previous numerical investigations have shown that specific architectures based on suppression or max units can successfully learn static or dynamic stimuli (situations). Here we provide a theoretical basis concerning the learning process convergence and the network response to a novel stimulus. We show that, besides learning “simple” static situations, a nD network can learn and replicate a sequence of up to n different vectors or frames. We find limits on the learning rate and show coupling matrices developing during training in different cases including expansion of the network into the case of nonlinear interunit coupling. Furthermore, we show that a specific coupling matrix provides low-pass-filter properties to the units, thus connecting networks constructed by static summation units with continuous-time networks. We also show under which conditions such networks can be used to perform arithmetic calculations by means of pattern completion.     

Oscillations and waves in the models of interactive neural populations

The dynamics of activity in interactive neural populations is simulated by the networks of Wilson–Cowan oscillators. Two extreme cases of connection architectures in the networks are considered: (1) 1D and 2D regular and homogeneous grids with local connections and (2) sparse random coupling. Propagating waves in the network have been found under the stationary external input and the regime of partial synchronization has been obtained for the periodic input. It has been shown that in the case of random coupling about 60% of neural populations demonstrate oscillatory activity and some of these oscillations are synchronous. The role of different types of dynamics in information processing is discussed. In particular, we discuss the regime of partial synchronization in the context of cortical microcircuits.     

Gene regulatory networks from multifactorial perturbations using Graphical Lasso: application to the DREAM4 challenge

A major challenge in the field of systems biology consists of predicting gene regulatory networks based on different training data. Within the DREAM4 initiative, we took part in the multifactorial sub-challenge that aimed to predict gene regulatory networks of size 100 from training data consisting of steady-state levels obtained after applying multifactorial perturbations to the original in silico network. Due to the static character of the challenge data, we tackled the problem via a sparse Gaussian Markov Random Field, which relates network topology with the covariance inverse generated by the gene measurements. As for the computations, we used the Graphical Lasso algorithm which provided a large range of candidate network topologies. The main task was to select the optimal network topology and for that, different model selection criteria were explored. The selected networks were compared with the golden standards and the results ranked using the scoring metrics applied in the challenge, giving a better insight in our submission and the way to improve it.Our approach provides an easy statistical and computational framework to infer gene regulatory networks that is suitable for large networks, even if the number of the observations (perturbations) is greater than the number of variables (genes).     

Modulation of the dynamics of cerebellar Purkinje cells through the interaction of excitatory and inhibitory feedforward pathways

The dynamics of cerebellar neuronal networks is controlled by the underlying building blocks of neurons and synapses between them. For which, the computation of Purkinje cells (PCs), the only output cells of the cerebellar cortex, is implemented through various types of neural pathways interactively routing excitation and inhibition converged to PCs. Such tuning of excitation and inhibition, coming from the gating of specific pathways as well as short-term plasticity (STP) of the synapses, plays a dominant role in controlling the PC dynamics in terms of firing rate and spike timing. PCs receive cascade feedforward inputs from two major neural pathways: the first one is the feedforward excitatory pathway from granule cells (GCs) to PCs; the second one is the feedforward inhibition pathway from GCs, via molecular layer interneurons (MLIs), to PCs. The GC-PC pathway, together with short-term dynamics of excitatory synapses, has been a focus over past decades, whereas recent experimental evidence shows that MLIs also greatly contribute to controlling PC activity. Therefore, it is expected that the diversity of excitation gated by STP of GC-PC synapses, modulated by strong inhibition from MLI-PC synapses, can promote the computation performed by PCs. However, it remains unclear how these two neural pathways are interacted to modulate PC dynamics. Here using a computational model of PC network installed with these two neural pathways, we addressed this question to investigate the change of PC firing dynamics at the level of single cell and network. We show that the nonlinear characteristics of excitatory STP dynamics can significantly modulate PC spiking dynamics mediated by inhibition. The changes in PC firing rate, firing phase, and temporal spike pattern, are strongly modulated by these two factors in different ways. MLIs mainly contribute to variable delays in the postsynaptic action potentials of PCs while modulated by excitation STP. Notably, the diversity of synchronization and pause response in the PC network is governed not only by the balance of excitation and inhibition, but also by the synaptic STP, depending on input burst patterns. Especially, the pause response shown in the PC network can only emerge with the interaction of both pathways. Together with other recent findings, our results show that the interaction of feedforward pathways of excitation and inhibition, incorporated with synaptic short-term dynamics, can dramatically regulate the PC activities that consequently change the network dynamics of the cerebellar circuit.     

Dynamics of Sparsely Connected Networks of Excitatory and Inhibitory Spiking Neurons

The dynamics of networks of sparsely connected excitatory and inhibitory integrate-and-fire neurons are studied analytically. The analysis reveals a rich repertoire of states, including synchronous states in which neurons fire regularly; asynchronous states with stationary global activity and very irregular individual cell activity; and states in which the global activity oscillates but individual cells fire irregularly, typically at rates lower than the global oscillation frequency. The network can switch between these states, provided the external frequency, or the balance between excitation and inhibition, is varied. Two types of network oscillations are observed. In the fast oscillatory state, the network frequency is almost fully controlled by the synaptic time scale. In the slow oscillatory state, the network frequency depends mostly on the membrane time constant. Finite size effects in the asynchronous state are also discussed.     

Asymptotic states and topological structure of an activation-deactivation chemical network

The influence of the topology on the asymptotic states of a network of interacting chemical species has been studied by simulating its time evolution. Random and scale-free networks have been designed to support relevant features of activation-deactivation reactions networks (mapping signal transduction networks) and the system of ordinary differential equations associated to the dynamics has been numerically solved. We analysed stationary states of the dynamics as a function of the network's connectivity and of the distribution of the chemical species on the network; we found important differences between the two topologies in the regime of low connectivity. In particular, only for low connected scale-free networks it is possible to find zero activity patterns as stationary states of the dynamics which work as signal off-states. Asymptotic features of random and scale-free networks become similar as the connectivity increases.     

Size-Independent Differences between the Mean of Discrete Stochastic Systems and the Corresponding Continuous Deterministic Systems

In this paper, it is shown that for a class of reaction networks, the discrete stochastic nature of the reacting species and reactions results in qualitative and quantitative differences between the mean of exact stochastic simulations and the prediction of the corresponding deterministic system. The differences are independent of the number of molecules of each species in the system under consideration. These reaction networks are open systems of chemical reactions with no zero-order reaction rates. They are characterized by at least two stationary points, one of which is a nonzero stable point, and one unstable trivial solution (stability based on a linear stability analysis of the deterministic system). Starting from a nonzero initial condition, the deterministic system never reaches the zero stationary point due to its unstable nature. In contrast, the result presented here proves that this zero-state is a stable stationary state for the discrete stochastic system, and other finite states have zero probability of existence at large times. This result generalizes previous theoretical studies and simulations of specific systems and provides a theoretical basis for analyzing a class of systems that exhibit such inconsistent behavior. This result has implications in the simulation of infection, apoptosis, and population kinetics, as it can be shown that for certain models the stochastic simulations will always yield different predictions for the mean behavior than the deterministic simulations.     

A decision-making Fokker–Planck model in computational neuroscience

In computational neuroscience, decision-making may be explained analyzing models based on the evolution of the average firing rates of two interacting neuron populations, e.g., in bistable visual perception problems. These models typically lead to a multi-stable scenario for the concerned dynamical systems. Nevertheless, noise is an important feature of the model taking into account both the finite-size effects and the decision’s robustness. These stochastic dynamical systems can be analyzed by studying carefully their associated Fokker–Planck partial differential equation. In particular, in the Fokker–Planck setting, we analytically discuss the asymptotic behavior for large times towards a unique probability distribution, and we propose a numerical scheme capturing this convergence. These simulations are used to validate deterministic moment methods recently applied to the stochastic differential system. Further, proving the existence, positivity and uniqueness of the probability density solution for the stationary equation, as well as for the time evolving problem, we show that this stabilization does happen. Finally, we discuss the convergence of the solution for large times to the stationary state. Our approach leads to a more detailed analytical and numerical study of decision-making models applied in computational neuroscience.