Event-driven simulation of neural population synchronization facilitated by electrical coupling

Most neural communication and processing tasks are driven by spikes. This has enabled the application of the event-driven simulation schemes. However the simulation of spiking neural networks based on complex models that cannot be simplified to analytical expressions (requiring numerical calculation) is very time consuming. Here we describe briefly an event-driven simulation scheme that uses pre-calculated table-based neuron characterizations to avoid numerical calculations during a network simulation, allowing the simulation of large-scale neural systems. More concretely we explain how electrical coupling can be simulated efficiently within this computation scheme, reproducing synchronization processes observed in detailed simulations of neural populations.     

Coarse-grained event tree analysis for quantifying Hodgkin-Huxley neuronal network dynamics

We present an event tree analysis of studying the dynamics of the Hodgkin-Huxley (HH) neuronal networks. Our study relies on a coarse-grained projection to event trees and to the event chains that comprise these trees by using a statistical collection of spatial-temporal sequences of relevant physiological observables (such as sequences of spiking multiple neurons). This projection can retain information about network dynamics that covers multiple features, swiftly and robustly. We demonstrate that for even small differences in inputs, some dynamical regimes of HH networks contain sufficiently higher order statistics as reflected in event chains within the event tree analysis. Therefore, this analysis is effective in discriminating small differences in inputs. Moreover, we use event trees to analyze the results computed from an efficient library-based numerical method proposed in our previous work, where a pre-computed high resolution data library of typical neuronal trajectories during the interval of an action potential (spike) allows us to avoid resolving the spikes in detail. In this way, we can evolve the HH networks using time steps one order of magnitude larger than the typical time steps used for resolving the trajectories without the library, while achieving comparable statistical accuracy in terms of average firing rate and power spectra of voltage traces. Our numerical simulation results show that the library method is efficient in the sense that the results generated by using this numerical method with much larger time steps contain sufficiently high order statistical structure of firing events that are similar to the ones obtained using a regular HH solver. We use our event tree analysis to demonstrate these statistical similarities.     

Spiking regularity in a noisy small-world neuronal network

The regularity of spiking oscillations is studied in the networks with different topological structures. The network is composed of coupled Fitz-Hugh-Nagumo neurons driven by colored noise. The investigation illustrates that the spike train in both the regular and the Watts-Strogatz small-world neuronal networks can show the best regularity at a moderate noise intensity, indicating the existence of coherence resonance. Moreover, the temporal coherence of the spike train in the small-world network is superior to that in a regular network due to the increase of the randomness of the network topology. Besides the noise intensity, the spiking regularity can be optimized by tuning the randomness of the network topological structure or by tuning the correlation time of the colored noise. In particular, under the cooperation of the small-world topology and the correlation time, the spike train with good regularity could sustain a large magnitude of the local noise.     

Spiking behavior and epileptiform oscillations in a discrete model of cortical neural networks

Summary We investigate the phenomenon of epileptiform activity using a discrete model of cortical neural networks. Our model is reduced to the elementary features of neurons and assumes simplified dynamics of action potentials and postsynaptic potentials. The discrete model provides a comparably high simulation speed which allows the rendering of phase diagrams and simulations of large neural networks in reasonable time. Further the reduction to the basic features of neurons provides insight into the essentials of a possible mechanism of epilepsy. Our computer simulations suggest that the detailed dynamics of postsynaptic and action potentials are not indispensable for obtaining epileptiform behavior on the system level. The simulation results of autonomously evolving networks exhibit a regime in which the network dynamics spontaneously switch between fluctuating and oscillating behavior and produce isolated network spikes without external stimulation. Inhibitory neurons have been found to play an important part in the synchronization of neural firing: an increased number of synapses established by inhibitory neurons onto other neurons induces a transition to the spiking regime. A decreased frequency accompanying the hypersynchronous population activity has only occurred with slow inhibitory postsynaptic potentials.     

Simulation of networks of spiking neurons: A review of tools and strategies

We review different aspects of the simulation of spiking neural networks. We start by reviewing the different types of simulation strategies and algorithms that are currently implemented. We next review the precision of those simulation strategies, in particular in cases where plasticity depends on the exact timing of the spikes. We overview different simulators and simulation environments presently available (restricted to those freely available, open source and documented). For each simulation tool, its advantages and pitfalls are reviewed, with an aim to allow the reader to identify which simulator is appropriate for a given task. Finally, we provide a series of benchmark simulations of different types of networks of spiking neurons, including Hodgkin–Huxley type, integrate-and-fire models, interacting with current-based or conductance-based synapses, using clock-driven or event-driven integration strategies. The same set of models are implemented on the different simulators, and the codes are made available. The ultimate goal of this review is to provide a resource to facilitate identifying the appropriate integration strategy and simulation tool to use for a given modeling problem related to spiking neural networks. Action Editor: Barry J. Richmond     

Emergence of preferred firing sequences in large spiking neural networks during simulated neuronal development

Two main processes concurrently refine the nervous system over the course of development: cell death and selective synaptic pruning. We simulated large spiking neural networks (100 x 100 neurons "at birth") characterized by an early developmental phase with cell death due to excessive firing rate, followed by the onset of spike timing dependent synaptic plasticity (STDP), driven by spatiotemporal patterns of stimulation. The cell death affected the inhibitory units more than the excitatory units during the early developmental phase. The network activity showed the appearance of recurrent spatiotemporal firing patterns along the STDP phase, thus suggesting the emergence of cell assemblies from the initially randomly connected networks. Some of these patterns were detected throughout the simulation despite the activity-driven network modifications while others disappeared.     

Neural dynamics as sampling: a model for stochastic computation in recurrent networks of spiking neurons

The organization of computations in networks of spiking neurons in the brain is still largely unknown, in particular in view of the inherently stochastic features of their firing activity and the experimentally observed trial-to-trial variability of neural systems in the brain. In principle there exists a powerful computational framework for stochastic computations, probabilistic inference by sampling, which can explain a large number of macroscopic experimental data in neuroscience and cognitive science. But it has turned out to be surprisingly difficult to create a link between these abstract models for stochastic computations and more detailed models of the dynamics of networks of spiking neurons. Here we create such a link and show that under some conditions the stochastic firing activity of networks of spiking neurons can be interpreted as probabilistic inference via Markov chain Monte Carlo (MCMC) sampling. Since common methods for MCMC sampling in distributed systems, such as Gibbs sampling, are inconsistent with the dynamics of spiking neurons, we introduce a different approach based on non-reversible Markov chains that is able to reflect inherent temporal processes of spiking neuronal activity through a suitable choice of random variables. We propose a neural network model and show by a rigorous theoretical analysis that its neural activity implements MCMC sampling of a given distribution, both for the case of discrete and continuous time. This provides a step towards closing the gap between abstract functional models of cortical computation and more detailed models of networks of spiking neurons.     

State transitions through inhibitory interneurons in a cortical network model

Inhibitory interneurons shape the spiking characteristics and computational properties of cortical networks. Interneuron subtypes can precisely regulate cortical function but the roles of interneuron subtypes for promoting different regimes of cortical activity remains unclear. Therefore, we investigated the impact of fast spiking and non-fast spiking interneuron subtypes on cortical activity using a network model with connectivity and synaptic properties constrained by experimental data. We found that network properties were more sensitive to modulation of the fast spiking population, with reductions of fast spiking excitability generating strong spike correlations and network oscillations. Paradoxically, reduced fast spiking excitability produced a reduction of global excitation-inhibition balance and features of an inhibition stabilised network, in which firing rates were driven by the activity of excitatory neurons within the network. Further analysis revealed that the synaptic interactions and biophysical features associated with fast spiking interneurons, in particular their rapid intrinsic response properties and short synaptic latency, enabled this state transition by enhancing gain within the excitatory population. Therefore, fast spiking interneurons may be uniquely positioned to control the strength of recurrent excitatory connectivity and the transition to an inhibition stabilised regime. Overall, our results suggest that interneuron subtypes can exert selective control over excitatory gain allowing for differential modulation of global network state.     

Joint monitoring and prediction of accrual and event times in clinical trials

In many clinical trials, the primary endpoint is time to an event of interest, for example, time to cardiac attack or tumor progression, and the statistical power of these trials is primarily driven by the number of events observed during the trials. In such trials, the number of events observed is impacted not only by the number of subjects enrolled but also by other factors including the event rate and the follow‐up duration. Consequently, it is important for investigators to be able to monitor and predict accurately patient accrual and event times so as to predict the times of interim and final analyses and enable efficient allocation of research resources, which have long been recognized as important aspects of trial design and conduct. The existing methods for prediction of event times all assume that patient accrual follows a Poisson process with a constant Poisson rate over time; however, it is fairly common in real‐life clinical trials that the Poisson rate changes over time. In this paper, we propose a Bayesian joint modeling approach for monitoring and prediction of accrual and event times in clinical trials. We employ a nonhomogeneous Poisson process to model patient accrual and a parametric or nonparametric model for the event and loss to follow‐up processes. Compared to existing methods, our proposed methods are more flexible and robust in that we model accrual and event/loss‐to‐follow‐up times jointly and allow the underlying accrual rates to change over time. We evaluate the performance of the proposed methods through simulation studies and illustrate the methods using data from a real oncology trial.     

Running as fast as it can: How spiking dynamics form object groupings in the laminar circuits of visual cortex

How spiking neurons cooperate to control behavioral processes is a fundamental problem in computational neuroscience. Such cooperative dynamics are required during visual perception when spatially distributed image fragments are grouped into emergent boundary contours. Perceptual grouping is a challenge for spiking cells because its properties of collinear facilitation and analog sensitivity occur in response to binary spikes with irregular timing across many interacting cells. Some models have demonstrated spiking dynamics in recurrent laminar neocortical circuits, but not how perceptual grouping occurs. Other models have analyzed the fast speed of certain percepts in terms of a single feedforward sweep of activity, but cannot explain other percepts, such as illusory contours, wherein perceptual ambiguity can take hundreds of milliseconds to resolve by integrating multiple spikes over time. The current model reconciles fast feedforward with slower feedback processing, and binary spikes with analog network-level properties, in a laminar cortical network of spiking cells whose emergent properties quantitatively simulate parametric data from neurophysiological experiments, including the formation of illusory contours; the structure of non-classical visual receptive fields; and self-synchronizing gamma oscillations. These laminar dynamics shed new light on how the brain resolves local informational ambiguities through the use of properly designed nonlinear feedback spiking networks which run as fast as they can, given the amount of uncertainty in the data that they process.     

Propagation and synchronization of reverberatory bursts in developing cultured networks

Developing networks of neural systems can exhibit spontaneous, synchronous activities called neural bursts, which can be important in the organization of functional neural circuits. Before the network matures, the activity level of a burst can reverberate in repeated rise-and-falls in periods of hundreds of milliseconds following an initial wave-like propagation of spiking activity, while the burst itself lasts for seconds. To investigate the spatiotemporal structure of the reverberatory bursts, we culture dissociated, rat cortical neurons on a high-density multi-electrode array to record the dynamics of neural activity over the growth and maturation of the network. We find the synchrony of the spiking significantly reduced following the initial wave and the activities become broadly distributed spatially. The synchrony recovers as the system reverberates until the end of the burst. Using a propagation model we infer the spreading speed of the spiking activity, which increases as the culture ages. We perform computer simulations of the system using a physiological model of spiking networks in two spatial dimensions and find the parameters that reproduce the observed resynchronization of spiking in the bursts. An analysis of the simulated dynamics suggests that the depletion of synaptic resources causes the resynchronization. The spatial propagation dynamics of the simulations match well with observations over the course of a burst and point to an interplay of the synaptic efficacy and the noisy neural self-activation in producing the morphology of the bursts.     

Democratic population decisions result in robust policy-gradient learning: a parametric study with GPU simulations

High performance computing on the Graphics Processing Unit (GPU) is an emerging field driven by the promise of high computational power at a low cost. However, GPU programming is a non-trivial task and moreover architectural limitations raise the question of whether investing effort in this direction may be worthwhile. In this work, we use GPU programming to simulate a two-layer network of Integrate-and-Fire neurons with varying degrees of recurrent connectivity and investigate its ability to learn a simplified navigation task using a policy-gradient learning rule stemming from Reinforcement Learning. The purpose of this paper is twofold. First, we want to support the use of GPUs in the field of Computational Neuroscience. Second, using GPU computing power, we investigate the conditions under which the said architecture and learning rule demonstrate best performance. Our work indicates that networks featuring strong Mexican-Hat-shaped recurrent connections in the top layer, where decision making is governed by the formation of a stable activity bump in the neural population (a "non-democratic" mechanism), achieve mediocre learning results at best. In absence of recurrent connections, where all neurons "vote" independently ("democratic") for a decision via population vector readout, the task is generally learned better and more robustly. Our study would have been extremely difficult on a desktop computer without the use of GPU programming. We present the routines developed for this purpose and show that a speed improvement of 5x up to 42x is provided versus optimised Python code. The higher speed is achieved when we exploit the parallelism of the GPU in the search of learning parameters. This suggests that efficient GPU programming can significantly reduce the time needed for simulating networks of spiking neurons, particularly when multiple parameter configurations are investigated.     

Firing-rate models for neurons with a broad repertoire of spiking behaviors

Capturing the response behavior of spiking neuron models with rate-based models facilitates the investigation of neuronal networks using powerful methods for rate-based network dynamics. To this end, we investigate the responses of two widely used neuron model types, the Izhikevich and augmented multi-adapative threshold (AMAT) models, to a range of spiking inputs ranging from step responses to natural spike data. We find (i) that linear-nonlinear firing rate models fitted to test data can be used to describe the firing-rate responses of AMAT and Izhikevich spiking neuron models in many cases; (ii) that firing-rate responses are generally too complex to be captured by first-order low-pass filters but require bandpass filters instead; (iii) that linear-nonlinear models capture the response of AMAT models better than of Izhikevich models; (iv) that the wide range of response types evoked by current-injection experiments collapses to few response types when neurons are driven by stationary or sinusoidally modulated Poisson input; and (v) that AMAT and Izhikevich models show different responses to spike input despite identical responses to current injections. Together, these findings suggest that rate-based models of network dynamics may capture a wider range of neuronal response properties by incorporating second-order bandpass filters fitted to responses of spiking model neurons. These models may contribute to bringing rate-based network modeling closer to the reality of biological neuronal networks.     

Periodic coupling strength-dependent multiple coherence resonance by time delay in Newman–Watts neuronal networks

Recently, multiple coherence resonance induced by time delay has been observed in neuronal networks with constant coupling strength. In this paper, by employing Newman–Watts Hodgkin–Huxley neuron networks with time-periodic coupling strength, we study how the temporal coherence of spiking behavior and coherence resonance by time delay change when the frequency of periodic coupling strength is varied. It is found that delay induced coherence resonance is dependent on periodic coupling strength and increases when the frequency of periodic coupling strength increases. Periodic coupling strength can also induce multiple coherence resonance, and the coherence resonance occurs when the frequency of periodic coupling strength is approximately multiple of the spiking frequency. These results show that for periodic coupling strength time delay can more frequently optimize the temporal coherence of spiking activity, and periodic coupling strength can repetitively optimize the temporal coherence of spiking activity as well. Frequency locking may be the mechanism for multiple coherence resonance induced by periodic coupling strength. These findings imply that periodic coupling strength is more efficient for enhancing the temporal coherence of spiking activity of neuronal networks, and thus it could play a more important role in improving the time precision of information processing and transmission in neural networks.     

A fast model of voltage-dependent NMDA receptors

NMDA receptors are among the crucial elements of central nervous system models. Recent studies show that both conductance and kinetics of these receptors are changing voltage-dependently in some parts of the brain. Therefore, several models have been introduced to simulate their current. However, on the one hand, kinetic models—which are able to simulate these voltage-dependent phenomena—are computationally expensive for modeling of large neural networks. On the other hand, classic exponential models, which are computationally less expensive, are not able to simulate the voltage-dependency of these receptors, accurately. In this study, we have modified these classic models to endow them with the voltage-dependent conductance and time constants. Temperature sensitivity and desensitization of these receptors are also taken into account. We show that, it is possible to simulate the most important physiological aspects of NMDA receptor’s behavior using only three to four differential equations, which is significantly smaller than the previous kinetic models. Consequently, it seems that our model is both fast and physiologically plausible and therefore is a suitable candidate for the modeling of large neural networks.     

Functional interactions in hierarchically organized neural networks studied with spatiotemporal firing patterns and phase-coupling frequencies

A scalable hardware/software hybrid module--called Ubidule--endowed with bio-inspired ontogenetic and epigenetic features is configured to run a neural networks simulation with developmental and evolvable capabilities. We simulated the activity of hierarchically organized spiking neural networks characterized by an initial developmental phase featuring cell death followed by spike timing dependent synaptic plasticity in presence of background noise. An upstream 'sensory' network received a spatiotemporally organized external input and downstream networks were activated only via the upstream network. Precise firing sequences, formed by recurrent patterns of spikes intervals above chance levels, were observed in all recording conditions, thus suggesting the build-up of a connectivity able to sustain temporal information processing. The activity of a Ubinet--a network of Ubidules--is analyzed by means of virtual electrodes that recorded neural signals similar to EEG. The analysis of these signals was compared with a small set of human recordings and revealed common patterns of shift in quadratic phase coupling. The results suggest some interpretations of changes and plasticity of functional interactions between cortical areas driven by external stimuli and by learning/cognitive     

Probabilistic inference in general graphical models through sampling in stochastic networks of spiking neurons

An important open problem of computational neuroscience is the generic organization of computations in networks of neurons in the brain. We show here through rigorous theoretical analysis that inherent stochastic features of spiking neurons, in combination with simple nonlinear computational operations in specific network motifs and dendritic arbors, enable networks of spiking neurons to carry out probabilistic inference through sampling in general graphical models. In particular, it enables them to carry out probabilistic inference in Bayesian networks with converging arrows ("explaining away") and with undirected loops, that occur in many real-world tasks. Ubiquitous stochastic features of networks of spiking neurons, such as trial-to-trial variability and spontaneous activity, are necessary ingredients of the underlying computational organization. We demonstrate through computer simulations that this approach can be scaled up to neural emulations of probabilistic inference in fairly large graphical models, yielding some of the most complex computations that have been carried out so far in networks of spiking neurons.     

Knowledge extraction from evolving spiking neural networks with rank order population coding

This paper demonstrates how knowledge can be extracted from evolving spiking neural networks with rank order population coding. Knowledge discovery is a very important feature of intelligent systems. Yet, a disproportionally small amount of research is centered on the issue of knowledge extraction from spiking neural networks which are considered to be the third generation of artificial neural networks. The lack of knowledge representation compatibility is becoming a major detriment to end users of these networks. We show that a high-level knowledge can be obtained from evolving spiking neural networks. More specifically, we propose a method for fuzzy rule extraction from an evolving spiking network with rank order population coding. The proposed method was used for knowledge discovery on two benchmark taste recognition problems where the knowledge learnt by an evolving spiking neural network was extracted in the form of zero-order Takagi-Sugeno fuzzy IF-THEN rules.