Finite-size and correlation-induced effects in mean-field dynamics

The brain’s activity is characterized by the interaction of a very large number of neurons that are strongly affected by noise. However, signals often arise at macroscopic scales integrating the effect of many neurons into a reliable pattern of activity. In order to study such large neuronal assemblies, one is often led to derive mean-field limits summarizing the effect of the interaction of a large number of neurons into an effective signal. Classical mean-field approaches consider the evolution of a deterministic variable, the mean activity, thus neglecting the stochastic nature of neural behavior. In this article, we build upon two recent approaches that include correlations and higher order moments in mean-field equations, and study how these stochastic effects influence the solutions of the mean-field equations, both in the limit of an infinite number of neurons and for large yet finite networks. We introduce a new model, the infinite model, which arises from both equations by a rescaling of the variables and, which is invertible for finite-size networks, and hence, provides equivalent equations to those previously derived models. The study of this model allows us to understand qualitative behavior of such large-scale networks. We show that, though the solutions of the deterministic mean-field equation constitute uncorrelated solutions of the new mean-field equations, the stability properties of limit cycles are modified by the presence of correlations, and additional non-trivial behaviors including periodic orbits appear when there were none in the mean field. The origin of all these behaviors is then explored in finite-size networks where interesting mesoscopic scale effects appear. This study leads us to show that the infinite-size system appears as a singular limit of the network equations, and for any finite network, the system will differ from the infinite system.     

Emergence of Slow-Switching Assemblies in Structured Neuronal Networks

Unraveling the interplay between connectivity and spatio-temporal dynamics in neuronal networks is a key step to advance our understanding of neuronal information processing. Here we investigate how particular features of network connectivity underpin the propensity of neural networks to generate slow-switching assembly (SSA) dynamics, i.e., sustained epochs of increased firing within assemblies of neurons which transition slowly between different assemblies throughout the network. We show that the emergence of SSA activity is linked to spectral properties of the asymmetric synaptic weight matrix. In particular, the leading eigenvalues that dictate the slow dynamics exhibit a gap with respect to the bulk of the spectrum, and the associated Schur vectors exhibit a measure of block-localization on groups of neurons, thus resulting in coherent dynamical activity on those groups. Through simple rate models, we gain analytical understanding of the origin and importance of the spectral gap, and use these insights to develop new network topologies with alternative connectivity paradigms which also display SSA activity. Specifically, SSA dynamics involving excitatory and inhibitory neurons can be achieved by modifying the connectivity patterns between both types of neurons. We also show that SSA activity can occur at multiple timescales reflecting a hierarchy in the connectivity, and demonstrate the emergence of SSA in small-world like networks. Our work provides a step towards understanding how network structure (uncovered through advancements in neuroanatomy and connectomics) can impact on spatio-temporal neural activity and constrain the resulting dynamics.     

The locust olfactory system as a case study for modeling dynamics of neurobiological networks: from discrete time neurons to continuous time neurons

Both chaotic and periodic activities are observed in networks of the central nervous systems. We choose the locust olfactory system as a good case study to analyze the relationships between networks' structure and the types of dynamics involved in coding mechanisms. In our modeling approach, we first build a fully connected recurrent network of synchronously updated McCulloch and Pitts neurons (MC-P type). In order to measure the use of the temporal dimension in the complex spatio-temporal patterns produced by the networks, we have defined an index the Normalized Euclidian Distance NED. We find that for appropriate parameters of input and connectivity, when adding some strong connections to the initial random synaptic matrices, it was easy to get the emergence of both robust oscillations and distributed synchrony in the spatiotemporal patterns. Then, in order to validate the MC-P model as a tool for analysis for network properties, we examine the dynamic behavior of networks of continuous time model neuron (Izhikevitch Integrate and Fire model -IFI-), implementing the same network characteristics. In both models, similarly to biological PN, the activity of excitatory neurons are phase-locked to different cycles of oscillations which remind the ones of the local field potential (LFP), and nevertheless exhibit dynamic behavior complex enough to be the basis of spatio-temporal codes.     

Oscillation and coding in a formal neural network considered as a guide for plausible simulations of the insect olfactory system

For the analysis of coding mechanisms in the insect olfactory system, a fully connected network of synchronously updated McCulloch and Pitts neurons (MC-P type) was developed [Quenet, B., Horn, D., 2003. The dynamic neural filter: a binary model of spatio-temporal coding. Neural Comput. 15 (2), 309-329]. Considering the update time as an intrinsic clock, this "Dynamic Neural Filter" (DNF), which maps regions of input space into spatio-temporal sequences of neuronal activity, is able to produce exact binary codes extracted from the synchronized activities recorded at the level of projection neurons (PN) in the locust antennal lobe (AL) in response to different odors [Wehr, M., Laurent, G., 1996. Odor encoding by temporal sequences of firing in oscillating neural assemblies. Nature 384, 162-166]. Here, in a first step, we separate the populations of PN and local inhibitory neurons (LN) and use the DNF as a guide for simulations based on biological plausible neurons (Hodgkin-Huxley: H-H type). We show that a parsimonious network of 10 H-H neurons generates action potentials whose timing represents the required codes. In a second step, we construct a new type of DNF in order to study the population dynamics when different delays are taken into account. We find synaptic matrices which lead to both the emergence of robust oscillations and spatio-temporal patterns, using a formal criterion, based on a Normalized Euclidian Distance (NED), in order to measure the use of the temporal dimension as a coding dimension by the DNF. Similarly to biological PN, the activity of excitatory neurons in the model can be both phase-locked to different cycles of oscillations which remind local field potential (LFP), and nevertheless exhibit dynamic behavior complex enough to be the basis of spatio-temporal codes.     

Pacemaker and network mechanisms of rhythm generation: cooperation and competition

The origin of rhythmic activity in brain circuits and CPG-like motor networks is still not fully understood. The main unsolved questions are (i) What are the respective roles of intrinsic bursting and network based dynamics in systems of coupled heterogeneous, intrinsically complex, even chaotic, neurons? (ii) What are the mechanisms underlying the coexistence of robustness and flexibility in the observed rhythmic spatio-temporal patterns? One common view is that particular bursting neurons provide the rhythmogenic component while the connections between different neurons are responsible for the regularisation and synchronisation of groups of neurons and for specific phase relationships in multi-phasic patterns. We have examined the spatio-temporal rhythmic patterns in computer-simulated motif networks of H-H neurons connected by slow inhibitory synapses with a non-symmetric pattern of coupling strengths. We demonstrate that the interplay between intrinsic and network dynamics features either cooperation or competition, depending on three basic control parameters identified in our model: the shape of intrinsic bursts, the strength of the coupling and its degree of asymmetry. The cooperation of intrinsic dynamics and network mechanisms is shown to correlate with bistability, i.e., the coexistence of two different attractors in the phase space of the system corresponding to different rhythmic spatio-temporal patterns. Conversely, if the network mechanism of rhythmogenesis dominates, monostability is observed with a typical pattern of winnerless competition between neurons. We analyse bifurcations between the two regimes and demonstrate how they provide robustness and flexibility to the network performance.     

Cortical Resonance Frequencies Emerge from Network Size and Connectivity

Neural oscillations occur within a wide frequency range with different brain regions exhibiting resonance-like characteristics at specific points in the spectrum. At the microscopic scale, single neurons possess intrinsic oscillatory properties, such that is not yet known whether cortical resonance is consequential to neural oscillations or an emergent property of the networks that interconnect them. Using a network model of loosely-coupled Wilson-Cowan oscillators to simulate a patch of cortical sheet, we demonstrate that the size of the activated network is inversely related to its resonance frequency. Further analysis of the parameter space indicated that the number of excitatory and inhibitory connections, as well as the average transmission delay between units, determined the resonance frequency. The model predicted that if an activated network within the visual cortex increased in size, the resonance frequency of the network would decrease. We tested this prediction experimentally using the steady-state visual evoked potential where we stimulated the visual cortex with different size stimuli at a range of driving frequencies. We demonstrate that the frequency corresponding to peak steady-state response inversely correlated with the size of the network. We conclude that although individual neurons possess resonance properties, oscillatory activity at the macroscopic level is strongly influenced by network interactions, and that the steady-state response can be used to investigate functional networks.     

Spatial firing patterns of auditory neuron network modelling by computer simulation

This communication examines, in digital computer simulated network, input signals and response patterns established at excitatory neurons' level i.e. the membrane potential of neuron soma. It is restricted to spatial patterns of the auditory neuron networks and time factor for nervous conduction and transmission is neglected compared with long maintained membrane potentials of neuron somas. The model analyzes the change in the spatial patterns of the membrane potential in the two dimensional networks of the auditory system. In order to evaluate the contribution of the various parameters, it is started that the simplest model has only one parameter, lateral inhibition. The other parameters are then added, one at a time, to successive models. The lateral inhibition is a necessary condition in the auditory nervous system if any sharpening of the response areas in the single neurons is to occur. A necessary condition for the validity of the model is that it should be applicable to the other senses such as vision and chemical patterns, taste. The threshold feature of auditory neurons aids in producing a sharpening in the neuron of the auditory relay nuclei. It does this clipping the spatial response patterns in one dimensional arrays of excitatory neurons. Recurrent inhibition seems a necessary condition in the sensory nervous system that any kinds of input signals are to be preserved over a wide range of stimulus intensity. In other words, this network has a wide dynamic range against any kinds of input signals. A simple self-recurrent negative feedback does not contribute to the sharpening, but more complex socalled averaged type does. A neuron network is capable of responding stably to stimuli with a wide range of intensity and with any kind of spatial patterns if there is a simple negative feedback mechanism. When there is no negative feedback, input signals soon disappear or saturate in the neuron network. Therefore, recurrent inhibition is the most important mechanism. Spontaneous activity appears to aid in the sharpening by providing a kind of contrast, that is by reducting the amount of activity in neurons adjacent to the excitatory area. Moreover, the effect of spontaneous activity in the model seems to make repples around the excitatory area and suggests that an introduction of activity at any stage of the networks, from whatever source for example reticulum formation and thalamus, might appreciably alter the response patterns at subsequent neuron network. This suggests that the mechanism of the consciousness that might be controlled by the thalamus and or reticular formation. These two dimensional neuron networks may be expanded to three dimensional neuron networks. The former might simulate the auditory nervous system while the latter might simulate the visual system.     

Children with well controlled epilepsy possess different spatio-temporal patterns of causal network connectivity during a visual working memory task

Using spectral Granger causality (GC) we identified distinct spatio-temporal causal connectivity (CC) patterns in groups of control and epileptic children during the execution of a one-back matching visual discrimination working memory task. Differences between control and epileptic groups were determined for both GO and NOGO conditions. The analysis was performed on a set of 19-channel EEG cortical activity signals. We show that for the GO task, the highest brain activity in terms of the density of the CC networks is observed in α band for the control group while for the epileptic group the CC network seems disrupted as reflected by the small number of connections. For the NOGO task, the denser CC network was observed in θ band for the control group while widespread differences between the control and the epileptic group were located bilaterally at the left temporal-midline and parietal areas. In order to test the discriminative power of our analysis, we performed a pattern analysis approach based on fuzzy classification techniques. The performance of the classification scheme was evaluated using permutation tests. The analysis demonstrated that, on average, 87.6 % of the subjects were correctly classified in control and epileptic. Thus, our findings may provide a helpful insight on the mechanisms pertaining to the cognitive response of children with well controlled epilepsy and could potentially serve as “functional” biomarkers for early diagnosis.     

Multi-Stability and Pattern-Selection in Oscillatory Networks with Fast Inhibition and Electrical Synapses

A model or hybrid network consisting of oscillatory cells interconnected by inhibitory and electrical synapses may express different stable activity patterns without any change of network topology or parameters, and switching between the patterns can be induced by specific transient signals. However, little is known of properties of such signals. In the present study, we employ numerical simulations of neural networks of different size composed of relaxation oscillators, to investigate switching between in-phase (IP) and anti-phase (AP) activity patterns. We show that the time windows of susceptibility to switching between the patterns are similar in 2-, 4- and 6-cell fully-connected networks. Moreover, in a network (N = 4, 6) expressing a given AP pattern, a stimulus with a given profile consisting of depolarizing and hyperpolarizing signals sent to different subpopulations of cells can evoke switching to another AP pattern. Interestingly, the resulting pattern encodes the profile of the switching stimulus. These results can be extended to different network architectures. Indeed, relaxation oscillators are not only models of cellular pacemakers, bursting or spiking, but are also analogous to firing-rate models of neural activity. We show that rules of switching similar to those found for relaxation oscillators apply to oscillating circuits of excitatory cells interconnected by electrical synapses and cross-inhibition. Our results suggest that incoming information, arriving in a proper time window, may be stored in an oscillatory network in the form of a specific spatio-temporal activity pattern which is expressed until new pertinent information arrives.     

Decorrelation of Neural-Network Activity by Inhibitory Feedback

Correlations in spike-train ensembles can seriously impair the encoding of information by their spatio-temporal structure. An inevitable source of correlation in finite neural networks is common presynaptic input to pairs of neurons. Recent studies demonstrate that spike correlations in recurrent neural networks are considerably smaller than expected based on the amount of shared presynaptic input. Here, we explain this observation by means of a linear network model and simulations of networks of leaky integrate-and-fire neurons. We show that inhibitory feedback efficiently suppresses pairwise correlations and, hence, population-rate fluctuations, thereby assigning inhibitory neurons the new role of active decorrelation. We quantify this decorrelation by comparing the responses of the intact recurrent network (feedback system) and systems where the statistics of the feedback channel is perturbed (feedforward system). Manipulations of the feedback statistics can lead to a significant increase in the power and coherence of the population response. In particular, neglecting correlations within the ensemble of feedback channels or between the external stimulus and the feedback amplifies population-rate fluctuations by orders of magnitude. The fluctuation suppression in homogeneous inhibitory networks is explained by a negative feedback loop in the one-dimensional dynamics of the compound activity. Similarly, a change of coordinates exposes an effective negative feedback loop in the compound dynamics of stable excitatory-inhibitory networks. The suppression of input correlations in finite networks is explained by the population averaged correlations in the linear network model: In purely inhibitory networks, shared-input correlations are canceled by negative spike-train correlations. In excitatory-inhibitory networks, spike-train correlations are typically positive. Here, the suppression of input correlations is not a result of the mere existence of correlations between excitatory (E) and inhibitory (I) neurons, but a consequence of a particular structure of correlations among the three possible pairings (EE, EI, II).     

Integrate-and-fire vs Poisson models of LGN input to V1 cortex: noisier inputs reduce orientation selectivity

One of the reasons the visual cortex has attracted the interest of computational neuroscience is that it has well-defined inputs. The lateral geniculate nucleus (LGN) of the thalamus is the source of visual signals to the primary visual cortex (V1). Most large-scale cortical network models approximate the spike trains of LGN neurons as simple Poisson point processes. However, many studies have shown that neurons in the early visual pathway are capable of spiking with high temporal precision and their discharges are not Poisson-like. To gain an understanding of how response variability in the LGN influences the behavior of V1, we study response properties of model V1 neurons that receive purely feedforward inputs from LGN cells modeled either as noisy leaky integrate-and-fire (NLIF) neurons or as inhomogeneous Poisson processes. We first demonstrate that the NLIF model is capable of reproducing many experimentally observed statistical properties of LGN neurons. Then we show that a V1 model in which the LGN input to a V1 neuron is modeled as a group of NLIF neurons produces higher orientation selectivity than the one with Poisson LGN input. The second result implies that statistical characteristics of LGN spike trains are important for V1’s function. We conclude that physiologically motivated models of V1 need to include more realistic LGN spike trains that are less noisy than inhomogeneous Poisson processes.     

Visual pattern recognition based on spatio-temporal patterns of retinal ganglion cells’ activities

Neural information is processed based on integrated activities of relevant neurons. Concerted population activity is one of the important ways for retinal ganglion cells to efficiently organize and process visual information. In the present study, the spike activities of bullfrog retinal ganglion cells in response to three different visual patterns (checker-board, vertical gratings and horizontal gratings) were recorded using multi-electrode arrays. A measurement of subsequence distribution discrepancy (MSDD) was applied to identify the spatio-temporal patterns of retinal ganglion cells' activities in response to different stimulation patterns. The results show that the population activity patterns were different in response to different stimulation patterns, such difference in activity pattern was consistently detectable even when visual adaptation occurred during repeated experimental trials. Therefore, the stimulus pattern can be reliably discriminated according to the spatio-temporal pattern of the neuronal activities calculated using the MSDD algorithm.     

Spatial constancy and the brain: insights from neural networks

To form an accurate internal representation of visual space, the brain must accurately account for movements of the eyes, head or body. Updating of internal representations in response to these movements is especially important when remembering spatial information, such as the location of an object, since the brain must rely on non-visual extra-retinal signals to compensate for self-generated movements. We investigated the computations underlying spatial updating by constructing a recurrent neural network model to store and update a spatial location based on a gaze shift signal, and to do so flexibly based on a contextual cue. We observed a striking similarity between the patterns of behaviour produced by the model and monkeys trained to perform the same task, as well as between the hidden units of the model and neurons in the lateral intraparietal area (LIP). In this report, we describe the similarities between the model and single unit physiology to illustrate the usefulness of neural networks as a tool for understanding specific computations performed by the brain.     

Noise-induced transitions in slow wave neuronal dynamics

Many neuronal systems exhibit slow random alternations and sudden switches in activity states. Models with noisy relaxation dynamics (oscillatory, excitable or bistable) account for these temporal, slow wave, patterns and the fluctuations within states. The noise-induced transitions in a relaxation dynamics are analogous to escape by a particle in a slowly changing double-well potential. In this formalism, we obtain semi-analytically the first and second order statistical properties: the distributions of the slow process at the transitions and the temporal correlations of successive switching events. We find that the temporal correlations can be used to help distinguish among biophysical mechanisms for the slow negative feedback, such as divisive or subtractive. We develop our results in the context of models for cellular pacemaker neurons; they also apply to mean-field models for spontaneously active networks with slow wave dynamics.     

Functional clustering in hippocampal cultures: relating network structure and dynamics

In this work we investigate the relationship between gross anatomic structural network properties, neuronal dynamics and the resultant functional structure in dissociated rat hippocampal cultures. Specifically, we studied cultures as they developed under two conditions: the first supporting glial cell growth (high glial group), and the second one inhibiting it (low glial group). We then compared structural network properties and the spatio-temporal activity patterns of the neurons. Differences in dynamics between the two groups could be linked to the impact of the glial network on the neuronal network as the cultures developed. We also implemented a recently developed algorithm called the functional clustering algorithm (FCA) to obtain the resulting functional network structure. We show that this new algorithm is useful for capturing changes in functional network structure as the networks evolve over time. The FCA detects changes in functional structure that are consistent with expected dynamical differences due to the impact of the glial network. Cultures in the high glial group show an increase in global synchronization as the cultures age, while those in the low glial group remain locally synchronized. We additionally use the FCA to quantify the amount of synchronization present in the cultures and show that the total level of synchronization in the high glial group is stronger than in the low glial group. These results indicate an interdependence between the glial and neuronal networks present in dissociated cultures.     

Linking dynamics of the inhibitory network to the input structure

Networks of inhibitory interneurons are found in many distinct classes of biological systems. Inhibitory interneurons govern the dynamics of principal cells and are likely to be critically involved in the coding of information. In this theoretical study, we describe the dynamics of a generic inhibitory network in terms of low-dimensional, simplified rate models. We study the relationship between the structure of external input applied to the network and the patterns of activity arising in response to that stimulation. We found that even a minimal inhibitory network can generate a great diversity of spatio-temporal patterning including complex bursting regimes with non-trivial ratios of burst firing. Despite the complexity of these dynamics, the network’s response patterns can be predicted from the rankings of the magnitudes of external inputs to the inhibitory neurons. This type of invariant dynamics is robust to noise and stable in densely connected networks with strong inhibitory coupling. Our study predicts that the response dynamics generated by an inhibitory network may provide critical insights about the temporal structure of the sensory input it receives.     

Network Adaptation Improves Temporal Representation of Naturalistic Stimuli in Drosophila Eye: I Dynamics

Because of the limited processing capacity of eyes, retinal networks must adapt constantly to best present the ever changing visual world to the brain. However, we still know little about how adaptation in retinal networks shapes neural encoding of changing information. To study this question, we recorded voltage responses from photoreceptors (R1–R6) and their output neurons (LMCs) in the Drosophila eye to repeated patterns of contrast values, collected from natural scenes. By analyzing the continuous photoreceptor-to-LMC transformations of these graded-potential neurons, we show that the efficiency of coding is dynamically improved by adaptation. In particular, adaptation enhances both the frequency and amplitude distribution of LMC output by improving sensitivity to under-represented signals within seconds. Moreover, the signal-to-noise ratio of LMC output increases in the same time scale. We suggest that these coding properties can be used to study network adaptation using the genetic tools in Drosophila, as shown in a companion paper (Part II).     

Discriminating signal from noise: recognition of a movement-based animal display by artificial neural networks

In this study, we investigated the feasibility of applying neural networks to understanding movement-based visual signals. Networks based on three different models were constructed, varying in their input format and network architecture: a Static Input model, a Dynamic Input model and a Feedback model. The task for all networks was to distinguish a lizard (Amphibolurus muricatus) tail-flick from background plant movement. Networks based on all models were able to distinguish the two types of visual motion, and generalised successfully to unseen exemplars. We used curves defined by the receiver-operating characteristic (ROC) to select a single network from each model to be used in regression analyses of network response and several motion variables. Collectively, the models predicted that tail-flick efficacy would be enhanced by faster speeds, greater acceleration and longer durations.