Firing-rate models for neurons with a broad repertoire of spiking behaviors

Capturing the response behavior of spiking neuron models with rate-based models facilitates the investigation of neuronal networks using powerful methods for rate-based network dynamics. To this end, we investigate the responses of two widely used neuron model types, the Izhikevich and augmented multi-adapative threshold (AMAT) models, to a range of spiking inputs ranging from step responses to natural spike data. We find (i) that linear-nonlinear firing rate models fitted to test data can be used to describe the firing-rate responses of AMAT and Izhikevich spiking neuron models in many cases; (ii) that firing-rate responses are generally too complex to be captured by first-order low-pass filters but require bandpass filters instead; (iii) that linear-nonlinear models capture the response of AMAT models better than of Izhikevich models; (iv) that the wide range of response types evoked by current-injection experiments collapses to few response types when neurons are driven by stationary or sinusoidally modulated Poisson input; and (v) that AMAT and Izhikevich models show different responses to spike input despite identical responses to current injections. Together, these findings suggest that rate-based models of network dynamics may capture a wider range of neuronal response properties by incorporating second-order bandpass filters fitted to responses of spiking model neurons. These models may contribute to bringing rate-based network modeling closer to the reality of biological neuronal networks.     

Spike-based population coding and working memory

Compelling behavioral evidence suggests that humans can make optimal decisions despite the uncertainty inherent in perceptual or motor tasks. A key question in neuroscience is how populations of spiking neurons can implement such probabilistic computations. In this article, we develop a comprehensive framework for optimal, spike-based sensory integration and working memory in a dynamic environment. We propose that probability distributions are inferred spike-per-spike in recurrently connected networks of integrate-and-fire neurons. As a result, these networks can combine sensory cues optimally, track the state of a time-varying stimulus and memorize accumulated evidence over periods much longer than the time constant of single neurons. Importantly, we propose that population responses and persistent working memory states represent entire probability distributions and not only single stimulus values. These memories are reflected by sustained, asynchronous patterns of activity which make relevant information available to downstream neurons within their short time window of integration. Model neurons act as predictive encoders, only firing spikes which account for new information that has not yet been signaled. Thus, spike times signal deterministically a prediction error, contrary to rate codes in which spike times are considered to be random samples of an underlying firing rate. As a consequence of this coding scheme, a multitude of spike patterns can reliably encode the same information. This results in weakly correlated, Poisson-like spike trains that are sensitive to initial conditions but robust to even high levels of external neural noise. This spike train variability reproduces the one observed in cortical sensory spike trains, but cannot be equated to noise. On the contrary, it is a consequence of optimal spike-based inference. In contrast, we show that rate-based models perform poorly when implemented with stochastically spiking neurons.     

Firing-rate models capture essential response dynamics of LGN relay cells

Firing-rate models provide a practical tool for studying signal processing in the early visual system, permitting more thorough mathematical analysis than spike-based models. We show here that essential response properties of relay cells in the lateral geniculate nucleus (LGN) can be captured by surprisingly simple firing-rate models consisting of a low-pass filter and a nonlinear activation function. The starting point for our analysis are two spiking neuron models based on experimental data: a spike-response model fitted to data from macaque (Carandini et al. J. Vis., 20(14), 1–2011, 2007), and a model with conductance-based synapses and afterhyperpolarizing currents fitted to data from cat (Casti et al. J. Comput. Neurosci., 24(2), 235–252, 2008). We obtained the nonlinear activation function by stimulating the model neurons with stationary stochastic spike trains, while we characterized the linear filter by fitting a low-pass filter to responses to sinusoidally modulated stochastic spike trains. To account for the non-Poisson nature of retinal spike trains, we performed all analyses with spike trains with higher-order gamma statistics in addition to Poissonian spike trains. Interestingly, the properties of the low-pass filter depend only on the average input rate, but not on the modulation depth of sinusoidally modulated input. Thus, the response properties of our model are fully specified by just three parameters (low-frequency gain, cutoff frequency, and delay) for a given mean input rate and input regularity. This simple firing-rate model reproduces the response of spiking neurons to a step in input rate very well for Poissonian as well as for non-Poissonian input. We also found that the cutoff frequencies, and thus the filter time constants, of the rate-based model are unrelated to the membrane time constants of the underlying spiking models, in agreement with similar observations for simpler models.     

Spiking Neurons in a Hierarchical Self-Organizing Map Model Can Learn to Develop Spatial and Temporal Properties of Entorhinal Grid Cells and Hippocampal Place Cells

Medial entorhinal grid cells and hippocampal place cells provide neural correlates of spatial representation in the brain. A place cell typically fires whenever an animal is present in one or more spatial regions, or places, of an environment. A grid cell typically fires in multiple spatial regions that form a regular hexagonal grid structure extending throughout the environment. Different grid and place cells prefer spatially offset regions, with their firing fields increasing in size along the dorsoventral axes of the medial entorhinal cortex and hippocampus. The spacing between neighboring fields for a grid cell also increases along the dorsoventral axis. This article presents a neural model whose spiking neurons operate in a hierarchy of self-organizing maps, each obeying the same laws. This spiking GridPlaceMap model simulates how grid cells and place cells may develop. It responds to realistic rat navigational trajectories by learning grid cells with hexagonal grid firing fields of multiple spatial scales and place cells with one or more firing fields that match neurophysiological data about these cells and their development in juvenile rats. The place cells represent much larger spaces than the grid cells, which enable them to support navigational behaviors. Both self-organizing maps amplify and learn to categorize the most frequent and energetic co-occurrences of their inputs. The current results build upon a previous rate-based model of grid and place cell learning, and thus illustrate a general method for converting rate-based adaptive neural models, without the loss of any of their analog properties, into models whose cells obey spiking dynamics. New properties of the spiking GridPlaceMap model include the appearance of theta band modulation. The spiking model also opens a path for implementation in brain-emulating nanochips comprised of networks of noisy spiking neurons with multiple-level adaptive weights for controlling autonomous adaptive robots capable of spatial navigation.     

Methods for short time series analysis of cell-based biosensor data.

This paper describes two approaches for sensing changes in spiking cells when only a limited amount of spike data is available, i.e., dynamically constructed local expansion rates and spike area distributions. The two methods were tested on time series from cultured neuron cells that exhibit spiking both autonomously and in the presence of periodic stimulation. Our tested hypothesis was that minute concentrations of toxins could affect the local statistics of the dynamics. Short data sets having relatively few spikes were generated from experiments on cells before and after being treated with a small concentration of channel blocker. In spontaneous spiking cells, local expansion rates show a sensitivity that correlates with channel concentration level, while stimulated cells show no such correlation. Spike area distributions on the other hand showed measurable differences between control and treated conditions for both types of spiking, and a much higher degree of sensitivity. Because these methods are based on analysis of short time series analysis, they might provide novel means for cell drug and toxin detection.     

Neural dynamics as sampling: a model for stochastic computation in recurrent networks of spiking neurons

The organization of computations in networks of spiking neurons in the brain is still largely unknown, in particular in view of the inherently stochastic features of their firing activity and the experimentally observed trial-to-trial variability of neural systems in the brain. In principle there exists a powerful computational framework for stochastic computations, probabilistic inference by sampling, which can explain a large number of macroscopic experimental data in neuroscience and cognitive science. But it has turned out to be surprisingly difficult to create a link between these abstract models for stochastic computations and more detailed models of the dynamics of networks of spiking neurons. Here we create such a link and show that under some conditions the stochastic firing activity of networks of spiking neurons can be interpreted as probabilistic inference via Markov chain Monte Carlo (MCMC) sampling. Since common methods for MCMC sampling in distributed systems, such as Gibbs sampling, are inconsistent with the dynamics of spiking neurons, we introduce a different approach based on non-reversible Markov chains that is able to reflect inherent temporal processes of spiking neuronal activity through a suitable choice of random variables. We propose a neural network model and show by a rigorous theoretical analysis that its neural activity implements MCMC sampling of a given distribution, both for the case of discrete and continuous time. This provides a step towards closing the gap between abstract functional models of cortical computation and more detailed models of networks of spiking neurons.     

Dynamic interactions between orexin and dynorphin may delay onset of functional orexin effects: a modeling study

Orexin (also known as hypocretin) neurons play a key role in regulating sleep-wake behavior, but the links between orexin neuron electrophysiology and function have not been explored. Orexin neurons are wake-active, and spiking activity in orexin neurons may anticipate transitions to wakefulness by several seconds. However, it is suggested that while the orexin system is necessary to maintain sustained wake bouts, orexin has little effect on brief wake bouts. In vitro experiments investigating the actions of orexin and dynorphin, a colocalized neuropeptide, on orexin neurons indicate that the dynamics of desensitization to dynorphin may represent a mechanism for modulating local network activity and resolving the apparent discrepancy between the onset of firing in orexin neurons and the onset of functional orexin effects. To investigate the role of dynorphin on orexin neuron activity, a Hodgkin-Huxley-type model orexin neuron was developed in which baseline electrophysiology, orexin/dynorphin action, and dynorphin desensitization were closely tied to experimental data. In this model framework, model orexin neuron responses to orexin/dynorphin action were calibrated by simulating the physiologic effects of static orexin and dynorphin bath application on orexin neurons. Then behavior in a small network of model orexin neurons was simulated with pure orexin, pure dynorphin, or combined orexin and dynorphin coupling based on the mechanisms established in the static case. It was found that the dynamics of desensitization to dynorphin can mediate a clear shift from a network in which firing is suppressed by dynorphin-mediated inhibition to a network of neurons with high firing rates sustained by orexin-mediated excitation. The findings suggest that dynamic interactions between orexin and dynorphin at transitions from sleep to wake may delay onset of functional orexin effects.     

Self-Organization of Microcircuits in Networks of Spiking Neurons with Plastic Synapses

The synaptic connectivity of cortical networks features an overrepresentation of certain wiring motifs compared to simple random-network models. This structure is shaped, in part, by synaptic plasticity that promotes or suppresses connections between neurons depending on their joint spiking activity. Frequently, theoretical studies focus on how feedforward inputs drive plasticity to create this network structure. We study the complementary scenario of self-organized structure in a recurrent network, with spike timing-dependent plasticity driven by spontaneous dynamics. We develop a self-consistent theory for the evolution of network structure by combining fast spiking covariance with a slow evolution of synaptic weights. Through a finite-size expansion of network dynamics we obtain a low-dimensional set of nonlinear differential equations for the evolution of two-synapse connectivity motifs. With this theory in hand, we explore how the form of the plasticity rule drives the evolution of microcircuits in cortical networks. When potentiation and depression are in approximate balance, synaptic dynamics depend on weighted divergent, convergent, and chain motifs. For additive, Hebbian STDP these motif interactions create instabilities in synaptic dynamics that either promote or suppress the initial network structure. Our work provides a consistent theoretical framework for studying how spiking activity in recurrent networks interacts with synaptic plasticity to determine network structure.     

Regulating synchronous oscillations of cerebellar granule cells by different types of inhibition

Synchronous oscillations in neural populations are considered being controlled by inhibitory neurons. In the granular layer of the cerebellum, two major types of cells are excitatory granular cells (GCs) and inhibitory Golgi cells (GoCs). GC spatiotemporal dynamics, as the output of the granular layer, is highly regulated by GoCs. However, there are various types of inhibition implemented by GoCs. With inputs from mossy fibers, GCs and GoCs are reciprocally connected to exhibit different network motifs of synaptic connections. From the view of GCs, feedforward inhibition is expressed as the direct input from GoCs excited by mossy fibers, whereas feedback inhibition is from GoCs via GCs themselves. In addition, there are abundant gap junctions between GoCs showing another form of inhibition. It remains unclear how these diverse copies of inhibition regulate neural population oscillation changes. Leveraging a computational model of the granular layer network, we addressed this question to examine the emergence and modulation of network oscillation using different types of inhibition. We show that at the network level, feedback inhibition is crucial to generate neural oscillation. When short-term plasticity was equipped on GoC-GC synapses, oscillations were largely diminished. Robust oscillations can only appear with additional gap junctions. Moreover, there was a substantial level of cross-frequency coupling in oscillation dynamics. Such a coupling was adjusted and strengthened by GoCs through feedback inhibition. Taken together, our results suggest that the cooperation of distinct types of GoC inhibition plays an essential role in regulating synchronous oscillations of the GC population. With GCs as the sole output of the granular network, their oscillation dynamics could potentially enhance the computational capability of downstream neurons.     

Synchronous bursts on scale-free neuronal networks with attractive and repulsive coupling

This paper investigates the dependence of synchronization transitions of bursting oscillations on the information transmission delay over scale-free neuronal networks with attractive and repulsive coupling. It is shown that for both types of coupling, the delay always plays a subtle role in either promoting or impairing synchronization. In particular, depending on the inherent oscillation period of individual neurons, regions of irregular and regular propagating excitatory fronts appear intermittently as the delay increases. These delay-induced synchronization transitions are manifested as well-expressed minima in the measure for spatiotemporal synchrony. For attractive coupling, the minima appear at every integer multiple of the average oscillation period, while for the repulsive coupling, they appear at every odd multiple of the half of the average oscillation period. The obtained results are robust to the variations of the dynamics of individual neurons, the system size, and the neuronal firing type. Hence, they can be used to characterize attractively or repulsively coupled scale-free neuronal networks with delays.     

Statistical-mechanical measure of stochastic spiking coherence in a population of inhibitory subthreshold neurons

By varying the noise intensity, we study stochastic spiking coherence (i.e., collective coherence between noise-induced neural spikings) in an inhibitory population of subthreshold neurons (which cannot fire spontaneously without noise). This stochastic spiking coherence may be well visualized in the raster plot of neural spikes. For a coherent case, partially-occupied "stripes" (composed of spikes and indicating collective coherence) are formed in the raster plot. This partial occupation occurs due to "stochastic spike skipping" which is well shown in the multi-peaked interspike interval histogram. The main purpose of our work is to quantitatively measure the degree of stochastic spiking coherence seen in the raster plot. We introduce a new spike-based coherence measure M ( s ) by considering the occupation pattern and the pacing pattern of spikes in the stripes. In particular, the pacing degree between spikes is determined in a statistical-mechanical way by quantifying the average contribution of (microscopic) individual spikes to the (macroscopic) ensemble-averaged global potential. This "statistical-mechanical" measure M ( s ) is in contrast to the conventional measures such as the "thermodynamic" order parameter (which concerns the time-averaged fluctuations of the macroscopic global potential), the "microscopic" correlation-based measure (based on the cross-correlation between the microscopic individual potentials), and the measures of precise spike timing (based on the peri-stimulus time histogram). In terms of M ( s ), we quantitatively characterize the stochastic spiking coherence, and find that M ( s ) reflects the degree of collective spiking coherence seen in the raster plot very well. Hence, the "statistical-mechanical" spike-based measure M ( s ) may be used usefully to quantify the degree of stochastic spiking coherence in a statistical-mechanical way.     

Hamiltonian energy and coexistence of hidden firing patterns from bidirectional coupling between two different neurons

In this paper, bidirectional-coupled neurons through an asymmetric electrical synapse are investigated. These coupled neurons involve 2D Hindmarsh–Rose (HR) and 2D FitzHugh–Nagumo (FN) neurons. The equilibria of the coupled neurons model are investigated, and their stabilities have revealed that, for some values of the electrical synaptic weight, the model under consideration can display either self-excited or hidden firing patterns. In addition, the hidden coexistence of chaotic bursting with periodic spiking, chaotic spiking with period spiking, chaotic bursting with a resting pattern, and the coexistence of chaotic spiking with a resting pattern are also found for some sets of electrical synaptic coupling. For all the investigated phenomena, the Hamiltonian energy of the model is computed. It enables the estimation of the amount of energy released during the transition between the various electrical activities. Pspice simulations are carried out based on the analog circuit of the coupled neurons to support our numerical results. Finally, an STM32F407ZE microcontroller development board is exploited for the digital implementation of the proposed coupled neurons model.     

Successful Reconstruction of a Physiological Circuit with Known Connectivity from Spiking Activity Alone

Identifying the structure and dynamics of synaptic interactions between neurons is the first step to understanding neural network dynamics. The presence of synaptic connections is traditionally inferred through the use of targeted stimulation and paired recordings or by post-hoc histology. More recently, causal network inference algorithms have been proposed to deduce connectivity directly from electrophysiological signals, such as extracellularly recorded spiking activity. Usually, these algorithms have not been validated on a neurophysiological data set for which the actual circuitry is known. Recent work has shown that traditional network inference algorithms based on linear models typically fail to identify the correct coupling of a small central pattern generating circuit in the stomatogastric ganglion of the crab Cancer borealis. In this work, we show that point process models of observed spike trains can guide inference of relative connectivity estimates that match the known physiological connectivity of the central pattern generator up to a choice of threshold. We elucidate the necessary steps to derive faithful connectivity estimates from a model that incorporates the spike train nature of the data. We then apply the model to measure changes in the effective connectivity pattern in response to two pharmacological interventions, which affect both intrinsic neural dynamics and synaptic transmission. Our results provide the first successful application of a network inference algorithm to a circuit for which the actual physiological synapses between neurons are known. The point process methodology presented here generalizes well to larger networks and can describe the statistics of neural populations. In general we show that advanced statistical models allow for the characterization of effective network structure, deciphering underlying network dynamics and estimating information-processing capabilities.     

Phase-Coherence Transitions and Communication in the Gamma Range between Delay-Coupled Neuronal Populations

Synchronization between neuronal populations plays an important role in information transmission between brain areas. In particular, collective oscillations emerging from the synchronized activity of thousands of neurons can increase the functional connectivity between neural assemblies by coherently coordinating their phases. This synchrony of neuronal activity can take place within a cortical patch or between different cortical regions. While short-range interactions between neurons involve just a few milliseconds, communication through long-range projections between different regions could take up to tens of milliseconds. How these heterogeneous transmission delays affect communication between neuronal populations is not well known. To address this question, we have studied the dynamics of two bidirectionally delayed-coupled neuronal populations using conductance-based spiking models, examining how different synaptic delays give rise to in-phase/anti-phase transitions at particular frequencies within the gamma range, and how this behavior is related to the phase coherence between the two populations at different frequencies. We have used spectral analysis and information theory to quantify the information exchanged between the two networks. For different transmission delays between the two coupled populations, we analyze how the local field potential and multi-unit activity calculated from one population convey information in response to a set of external inputs applied to the other population. The results confirm that zero-lag synchronization maximizes information transmission, although out-of-phase synchronization allows for efficient communication provided the coupling delay, the phase lag between the populations, and the frequency of the oscillations are properly matched.     

Graded,Dynamically Routable Information Processing with Synfire-Gated Synfire Chains

Coherent neural spiking and local field potentials are believed to be signatures of the binding and transfer of information in the brain. Coherent activity has now been measured experimentally in many regions of mammalian cortex. Recently experimental evidence has been presented suggesting that neural information is encoded and transferred in packets, i.e., in stereotypical, correlated spiking patterns of neural activity. Due to their relevance to coherent spiking, synfire chains are one of the main theoretical constructs that have been appealed to in order to describe coherent spiking and information transfer phenomena. However, for some time, it has been known that synchronous activity in feedforward networks asymptotically either approaches an attractor with fixed waveform and amplitude, or fails to propagate. This has limited the classical synfire chain’s ability to explain graded neuronal responses. Recently, we have shown that pulse-gated synfire chains are capable of propagating graded information coded in mean population current or firing rate amplitudes. In particular, we showed that it is possible to use one synfire chain to provide gating pulses and a second, pulse-gated synfire chain to propagate graded information. We called these circuits synfire-gated synfire chains (SGSCs). Here, we present SGSCs in which graded information can rapidly cascade through a neural circuit, and show a correspondence between this type of transfer and a mean-field model in which gating pulses overlap in time. We show that SGSCs are robust in the presence of variability in population size, pulse timing and synaptic strength. Finally, we demonstrate the computational capabilities of SGSC-based information coding by implementing a self-contained, spike-based, modular neural circuit that is triggered by streaming input, processes the input, then makes a decision based on the processed information and shuts itself down.     

Functional Connectivity and Tuning Curves in Populations of Simultaneously Recorded Neurons

How interactions between neurons relate to tuned neural responses is a longstanding question in systems neuroscience. Here we use statistical modeling and simultaneous multi-electrode recordings to explore the relationship between these interactions and tuning curves in six different brain areas. We find that, in most cases, functional interactions between neurons provide an explanation of spiking that complements and, in some cases, surpasses the influence of canonical tuning curves. Modeling functional interactions improves both encoding and decoding accuracy by accounting for noise correlations and features of the external world that tuning curves fail to capture. In cortex, modeling coupling alone allows spikes to be predicted more accurately than tuning curve models based on external variables. These results suggest that statistical models of functional interactions between even relatively small numbers of neurons may provide a useful framework for examining neural coding.     

Distribution of correlated spiking events in a population-based approach for Integrate-and-Fire networks

Randomly connected populations of spiking neurons display a rich variety of dynamics. However, much of the current modeling and theoretical work has focused on two dynamical extremes: on one hand homogeneous dynamics characterized by weak correlations between neurons, and on the other hand total synchrony characterized by large populations firing in unison. In this paper we address the conceptual issue of how to mathematically characterize the partially synchronous “multiple firing events” (MFEs) which manifest in between these two dynamical extremes. We further develop a geometric method for obtaining the distribution of magnitudes of these MFEs by recasting the cascading firing event process as a first-passage time problem, and deriving an analytical approximation of the first passage time density valid for large neuron populations. Thus, we establish a direct link between the voltage distributions of excitatory and inhibitory neurons and the number of neurons firing in an MFE that can be easily integrated into population–based computational methods, thereby bridging the gap between homogeneous firing regimes and total synchrony.     

Dynamics of Competition between Subnetworks of Spiking Neuronal Networks in the Balanced State

We explore and analyze the nonlinear switching dynamics of neuronal networks with non-homogeneous connectivity. The general significance of such transient dynamics for brain function is unclear; however, for instance decision-making processes in perception and cognition have been implicated with it. The network under study here is comprised of three subnetworks of either excitatory or inhibitory leaky integrate-and-fire neurons, of which two are of the same type. The synaptic weights are arranged to establish and maintain a balance between excitation and inhibition in case of a constant external drive. Each subnetwork is randomly connected, where all neurons belonging to a particular population have the same in-degree and the same out-degree. Neurons in different subnetworks are also randomly connected with the same probability; however, depending on the type of the pre-synaptic neuron, the synaptic weight is scaled by a factor. We observed that for a certain range of the “within” versus “between” connection weights (bifurcation parameter), the network activation spontaneously switches between the two sub-networks of the same type. This kind of dynamics has been termed “winnerless competition”, which also has a random component here. In our model, this phenomenon is well described by a set of coupled stochastic differential equations of Lotka-Volterra type that imply a competition between the subnetworks. The associated mean-field model shows the same dynamical behavior as observed in simulations of large networks comprising thousands of spiking neurons. The deterministic phase portrait is characterized by two attractors and a saddle node, its stochastic component is essentially given by the multiplicative inherent noise of the system. We find that the dwell time distribution of the active states is exponential, indicating that the noise drives the system randomly from one attractor to the other. A similar model for a larger number of populations might suggest a general approach to study the dynamics of interacting populations of spiking networks.     

Associative memory of phase-coded spatiotemporal patterns in leaky Integrate and Fire networks

We study the collective dynamics of a Leaky Integrate and Fire network in which precise relative phase relationship of spikes among neurons are stored, as attractors of the dynamics, and selectively replayed at different time scales. Using an STDP-based learning process, we store in the connectivity several phase-coded spike patterns, and we find that, depending on the excitability of the network, different working regimes are possible, with transient or persistent replay activity induced by a brief signal. We introduce an order parameter to evaluate the similarity between stored and recalled phase-coded pattern, and measure the storage capacity. Modulation of spiking thresholds during replay changes the frequency of the collective oscillation or the number of spikes per cycle, keeping preserved the phases relationship. This allows a coding scheme in which phase, rate and frequency are dissociable. Robustness with respect to noise and heterogeneity of neurons parameters is studied, showing that, since dynamics is a retrieval process, neurons preserve stable precise phase relationship among units, keeping a unique frequency of oscillation, even in noisy conditions and with heterogeneity of internal parameters of the units.