A review of the integrate-and-fire neuron model: II. Inhomogeneous synaptic input and network properties

The integrate-and-fire neuron model describes the state of a neuron in terms of its membrane potential, which is determined by the synaptic inputs and the injected current that the neuron receives. When the membrane potential reaches a threshold, an action potential (spike) is generated. This review considers the model in which the synaptic input varies periodically and is described by an inhomogeneous Poisson process, with both current and conductance synapses. The focus is on the mathematical methods that allow the output spike distribution to be analyzed, including first passage time methods and the Fokker–Planck equation. Recent interest in the response of neurons to periodic input has in part arisen from the study of stochastic resonance, which is the noise-induced enhancement of the signal-to-noise ratio. Networks of integrate-and-fire neurons behave in a wide variety of ways and have been used to model a variety of neural, physiological, and psychological phenomena. The properties of the integrate-and-fire neuron model with synaptic input described as a temporally homogeneous Poisson process are reviewed in an accompanying paper (Burkitt in Biol Cybern, 2006).     

An extended model for a spiking neuron class

This paper proposes an extension to the model of a spiking neuron for information processing in artificial neural networks, developing a new approach for the dynamic threshold of the integrate-and-fire neuron. This new approach invokes characteristics of biological neurons such as the behavior of chemical synapses and the receptor field. We demonstrate how such a digital model of spiking neurons can solve complex nonlinear classification with a single neuron, performing experiments for the classical XOR problem. Compared with rate-coded networks and the classical integrate-and-fire model, the trained network demonstrated faster information processing, requiring fewer neurons and shorter learning periods. The extended model validates all the logic functions of biological neurons when such functions are necessary for the proper flow of binary codes through a neural network.     

Emergence of resonances in neural systems: the interplay between adaptive threshold and short-term synaptic plasticity

In this work we study the detection of weak stimuli by spiking (integrate-and-fire) neurons in the presence of certain level of noisy background neural activity. Our study has focused in the realistic assumption that the synapses in the network present activity-dependent processes, such as short-term synaptic depression and facilitation. Employing mean-field techniques as well as numerical simulations, we found that there are two possible noise levels which optimize signal transmission. This new finding is in contrast with the classical theory of stochastic resonance which is able to predict only one optimal level of noise. We found that the complex interplay between adaptive neuron threshold and activity-dependent synaptic mechanisms is responsible for this new phenomenology. Our main results are confirmed by employing a more realistic FitzHugh-Nagumo neuron model, which displays threshold variability, as well as by considering more realistic stochastic synaptic models and realistic signals such as poissonian spike trains.     

Emerging methodologies for the study of hypothalamic gonadotropin-releasing-hormone (GnRH) neurons

Gonadotropin-releasing-hormone (GnRH) neurons form part of a central neural oscillator that controls sexual reproduction through intermittent release of the GnRH peptide. Activity of GnRH neurons, and by extension release of GnRH, has been proposed to reflect intrinsic properties and synaptic input of GnRH neurons. To study GnRH neurons, we used traditional electrophysiology and computational methods. These emerging methodologies enhance the elucidation of processing in GnRH neurons. We used dynamic current-clamping to understand how living GnRH somata process input from glutamate and GABA, two key neurotransmitters in the neuroendocrine hypothalamus. In order to study the impact of synaptic integration in dendrites and neuronal morphology, we have developed full-morphology models of GnRH neurons. Using dynamic clamping, we have demonstrated that small-amplitude glutamatergic currents can drive repetitive firing in GnRH neurons. Furthermore, application of simulated GABAergic synapses with a depolarized reversal potential have revealed two functional subpopulations of GnRH neurons: one population in which GABA chronically depolarizes membrane potential (without inducing action potentials) and a second population in which GABAergic excitation results in slow spiking. Finally, when AMPA-type and GABA-type simulated inputs are applied together, action potentials occur when the AMPA-type conductance occurs during the descending phase of GABAergic excitation and at the nadir of GABAergic inhibition. Compartmental computer models have shown that excitatory synapses at >300 microns from somtata are unable to drive spiking with purely passive dendrites. In models with active dendrites, distal synapses are more efficient at driving spiking than somatic inputs. We then used our models to extend the results from dynamic current clamping at GnRH somata to distribute synaptic inputs along the dendrite. We show that propagation delays for dendritic synapses alter synaptic integration in GnRH neurons by widening the temporal window of interaction for the generation of action potentials. Finally, we have shown that changes in dendrite morphology can modulate the output of GnRH neurons by altering the efficacy of action potential generation in response to after-depolarization potentials (ADPs). Taken together, the methodologies of dynamic current clamping and multi-compartmental modeling can make major contributions to the study of synaptic integration and structure-function relationships in hypothalamic GnRH neurons. Use of these methodological approaches will continue to provide keen insights leading to conceptual advances in our understanding of reproductive hormone secretion in normal and pathological physiology and open the door to understanding whether the mechanisms of pulsatile GnRH release are conserved across species.     

The Influence of Synaptic Weight Distribution on Neuronal Population Dynamics

The manner in which different distributions of synaptic weights onto cortical neurons shape their spiking activity remains open. To characterize a homogeneous neuronal population, we use the master equation for generalized leaky integrate-and-fire neurons with shot-noise synapses. We develop fast semi-analytic numerical methods to solve this equation for either current or conductance synapses, with and without synaptic depression. We show that its solutions match simulations of equivalent neuronal networks better than those of the Fokker-Planck equation and we compute bounds on the network response to non-instantaneous synapses. We apply these methods to study different synaptic weight distributions in feed-forward networks. We characterize the synaptic amplitude distributions using a set of measures, called tail weight numbers, designed to quantify the preponderance of very strong synapses. Even if synaptic amplitude distributions are equated for both the total current and average synaptic weight, distributions with sparse but strong synapses produce higher responses for small inputs, leading to a larger operating range. Furthermore, despite their small number, such synapses enable the network to respond faster and with more stability in the face of external fluctuations.     

The response of cortical neurons to in vivo-like input current: theory and experiment: II. Time-varying and spatially distributed inputs

The response of a population of neurons to time-varying synaptic inputs can show a rich phenomenology, hardly predictable from the dynamical properties of the membrane’s inherent time constants. For example, a network of neurons in a state of spontaneous activity can respond significantly more rapidly than each single neuron taken individually. Under the assumption that the statistics of the synaptic input is the same for a population of similarly behaving neurons (mean field approximation), it is possible to greatly simplify the study of neural circuits, both in the case in which the statistics of the input are stationary (reviewed in La Camera et al. in Biol Cybern, 2008) and in the case in which they are time varying and unevenly distributed over the dendritic tree. Here, we review theoretical and experimental results on the single-neuron properties that are relevant for the dynamical collective behavior of a population of neurons. We focus on the response of integrate-and-fire neurons and real cortical neurons to long-lasting, noisy, in vivo-like stationary inputs and show how the theory can predict the observed rhythmic activity of cultures of neurons. We then show how cortical neurons adapt on multiple time scales in response to input with stationary statistics in vitro. Next, we review how it is possible to study the general response properties of a neural circuit to time-varying inputs by estimating the response of single neurons to noisy sinusoidal currents. Finally, we address the dendrite–soma interactions in cortical neurons leading to gain modulation and spike bursts, and show how these effects can be captured by a two-compartment integrate-and-fire neuron. Most of the experimental results reviewed in this article have been successfully reproduced by simple integrate-and-fire model neurons.     

From spiking neuron models to linear-nonlinear models

Neurons transform time-varying inputs into action potentials emitted stochastically at a time dependent rate. The mapping from current input to output firing rate is often represented with the help of phenomenological models such as the linear-nonlinear (LN) cascade, in which the output firing rate is estimated by applying to the input successively a linear temporal filter and a static non-linear transformation. These simplified models leave out the biophysical details of action potential generation. It is not a priori clear to which extent the input-output mapping of biophysically more realistic, spiking neuron models can be reduced to a simple linear-nonlinear cascade. Here we investigate this question for the leaky integrate-and-fire (LIF), exponential integrate-and-fire (EIF) and conductance-based Wang-Buzsáki models in presence of background synaptic activity. We exploit available analytic results for these models to determine the corresponding linear filter and static non-linearity in a parameter-free form. We show that the obtained functions are identical to the linear filter and static non-linearity determined using standard reverse correlation analysis. We then quantitatively compare the output of the corresponding linear-nonlinear cascade with numerical simulations of spiking neurons, systematically varying the parameters of input signal and background noise. We find that the LN cascade provides accurate estimates of the firing rates of spiking neurons in most of parameter space. For the EIF and Wang-Buzsáki models, we show that the LN cascade can be reduced to a firing rate model, the timescale of which we determine analytically. Finally we introduce an adaptive timescale rate model in which the timescale of the linear filter depends on the instantaneous firing rate. This model leads to highly accurate estimates of instantaneous firing rates.     

An adaptive visual neuronal model implementing competitive, temporally asymmetric Hebbian learning

A novel depth-from-motion vision model based on leaky integrate-and-fire (I&F) neurons incorporates the implications of recent neurophysiological findings into an algorithm for object discovery and depth analysis. Pulse-coupled I&F neurons capture the edges in an optical flow field and the associated time of travel of those edges is encoded as the neuron parameters, mainly the time constant of the membrane potential and synaptic weight. Correlations between spikes and their timing thus code depth in the visual field. Neurons have multiple output synapses connecting to neighbouring neurons with an initial Gaussian weight distribution. A temporally asymmetric learning rule is used to adapt the synaptic weights online, during which competitive behaviour emerges between the different input synapses of a neuron. It is shown that the competition mechanism can further improve the model performance. After training, the weights of synapses sourced from a neuron do not display a Gaussian distribution, having adapted to encode features of the scenes to which they have been exposed.     

Spike timing precision of neuronal circuits

Spike timing is believed to be a key factor in sensory information encoding and computations performed by the neurons and neuronal circuits. However, the considerable noise and variability, arising from the inherently stochastic mechanisms that exist in the neurons and the synapses, degrade spike timing precision. Computational modeling can help decipher the mechanisms utilized by the neuronal circuits in order to regulate timing precision. In this paper, we utilize semi-analytical techniques, which were adapted from previously developed methods for electronic circuits, for the stochastic characterization of neuronal circuits. These techniques, which are orders of magnitude faster than traditional Monte Carlo type simulations, can be used to directly compute the spike timing jitter variance, power spectral densities, correlation functions, and other stochastic characterizations of neuronal circuit operation. We consider three distinct neuronal circuit motifs: Feedback inhibition, synaptic integration, and synaptic coupling. First, we show that both the spike timing precision and the energy efficiency of a spiking neuron are improved with feedback inhibition. We unveil the underlying mechanism through which this is achieved. Then, we demonstrate that a neuron can improve on the timing precision of its synaptic inputs, coming from multiple sources, via synaptic integration: The phase of the output spikes of the integrator neuron has the same variance as that of the sample average of the phases of its inputs. Finally, we reveal that weak synaptic coupling among neurons, in a fully connected network, enables them to behave like a single neuron with a larger membrane area, resulting in an improvement in the timing precision through cooperation.     

Noise Suppression and Surplus Synchrony by Coincidence Detection

The functional significance of correlations between action potentials of neurons is still a matter of vivid debate. In particular, it is presently unclear how much synchrony is caused by afferent synchronized events and how much is intrinsic due to the connectivity structure of cortex. The available analytical approaches based on the diffusion approximation do not allow to model spike synchrony, preventing a thorough analysis. Here we theoretically investigate to what extent common synaptic afferents and synchronized inputs each contribute to correlated spiking on a fine temporal scale between pairs of neurons. We employ direct simulation and extend earlier analytical methods based on the diffusion approximation to pulse-coupling, allowing us to introduce precisely timed correlations in the spiking activity of the synaptic afferents. We investigate the transmission of correlated synaptic input currents by pairs of integrate-and-fire model neurons, so that the same input covariance can be realized by common inputs or by spiking synchrony. We identify two distinct regimes: In the limit of low correlation linear perturbation theory accurately determines the correlation transmission coefficient, which is typically smaller than unity, but increases sensitively even for weakly synchronous inputs. In the limit of high input correlation, in the presence of synchrony, a qualitatively new picture arises. As the non-linear neuronal response becomes dominant, the output correlation becomes higher than the total correlation in the input. This transmission coefficient larger unity is a direct consequence of non-linear neural processing in the presence of noise, elucidating how synchrony-coded signals benefit from these generic properties present in cortical networks.     

Learning in spiking neural networks by reinforcement of stochastic synaptic transmission

It is well-known that chemical synaptic transmission is an unreliable process, but the function of such unreliability remains unclear. Here I consider the hypothesis that the randomness of synaptic transmission is harnessed by the brain for learning, in analogy to the way that genetic mutation is utilized by Darwinian evolution. This is possible if synapses are "hedonistic," responding to a global reward signal by increasing their probabilities of vesicle release or failure, depending on which action immediately preceded reward. Hedonistic synapses learn by computing a stochastic approximation to the gradient of the average reward. They are compatible with synaptic dynamics such as short-term facilitation and depression and with the intricacies of dendritic integration and action potential generation. A network of hedonistic synapses can be trained to perform a desired computation by administering reward appropriately, as illustrated here through numerical simulations of integrate-and-fire model neurons.     

The response of cortical neurons to in vivo-like input current: theory and experiment

The study of several aspects of the collective dynamics of interacting neurons can be highly simplified if one assumes that the statistics of the synaptic input is the same for a large population of similarly behaving neurons (mean field approach). In particular, under such an assumption, it is possible to determine and study all the equilibrium points of the network dynamics when the neuronal response to noisy, in vivo-like, synaptic currents is known. The response function can be computed analytically for simple integrate-and-fire neuron models and it can be measured directly in experiments in vitro. Here we review theoretical and experimental results about the neural response to noisy inputs with stationary statistics. These response functions are important to characterize the collective neural dynamics that are proposed to be the neural substrate of working memory, decision making and other cognitive functions. Applications to the case of time-varying inputs are reviewed in a companion paper (Giugliano et al. in Biol Cybern, 2008). We conclude that modified integrate-and-fire neuron models are good enough to reproduce faithfully many of the relevant dynamical aspects of the neuronal response measured in experiments on real neurons in vitro.     

A Statistical Model for In Vivo Neuronal Dynamics

Single neuron models have a long tradition in computational neuroscience. Detailed biophysical models such as the Hodgkin-Huxley model as well as simplified neuron models such as the class of integrate-and-fire models relate the input current to the membrane potential of the neuron. Those types of models have been extensively fitted to in vitro data where the input current is controlled. Those models are however of little use when it comes to characterize intracellular in vivo recordings since the input to the neuron is not known. Here we propose a novel single neuron model that characterizes the statistical properties of in vivo recordings. More specifically, we propose a stochastic process where the subthreshold membrane potential follows a Gaussian process and the spike emission intensity depends nonlinearly on the membrane potential as well as the spiking history. We first show that the model has a rich dynamical repertoire since it can capture arbitrary subthreshold autocovariance functions, firing-rate adaptations as well as arbitrary shapes of the action potential. We then show that this model can be efficiently fitted to data without overfitting. We finally show that this model can be used to characterize and therefore precisely compare various intracellular in vivo recordings from different animals and experimental conditions.     

The Possible Role of Spike Patterns in Cortical Information Processing

When the same visual stimulus is presented across many trials, neurons in the visual cortex receive stimulus-related synaptic inputs that are reproducible across trials (S) and inputs that are not (N). The variability of spike trains recorded in the visual cortex and their apparent lack of spike-to-spike correlations beyond that implied by firing rate fluctuations, has been taken as evidence for a low S/N ratio. A recent re-analysis of in vivo cortical data revealed evidence for spike-to-spike correlations in the form of spike patterns. We examine neural dynamics at a higher S/N in order to determine what possible role spike patterns could play in cortical information processing. In vivo-like spike patterns were obtained in model simulations. Superpositions of multiple sinusoidal driving currents were especially effective in producing stable long-lasting patterns. By applying current pulses that were either short and strong or long and weak, neurons could be made to switch from one pattern to another. Cortical neurons with similar stimulus preferences are located near each other, have similar biophysical properties and receive a large number of common synaptic inputs. Hence, recordings of a single neuron across multiple trials are usually interpreted as the response of an ensemble of these neurons during one trial. In the presence of distinct spike patterns across trials there is ambiguity in what would be the corresponding ensemble, it could consist of the same spike pattern for each neuron or a set of patterns across neurons. We found that the spiking response of a neuron receiving these ensemble inputs was determined by the spike-pattern composition, which, in turn, could be modulated dynamically as a means for cortical information processing.     

Synchronization of an Excitatory Integrate-and-Fire Neural Network

In this paper, we study the influence of the coupling strength on the synchronization behavior of a population of leaky integrate-and-fire neurons that is self-excitatory with a population density approach. Each neuron of the population is assumed to be stochastically driven by an independent Poisson spike train and the synaptic interaction between neurons is modeled by a potential jump at the reception of an action potential. Neglecting the synaptic delay, we will establish that for a strong enough connectivity between neurons, the solution of the partial differential equation which describes the population density function must blow up in finite time. Furthermore, we will give a mathematical estimate on the average connection per neuron to ensure the occurrence of a burst. Interpreting the blow up of the solution as the presence of a Dirac mass in the firing rate of the population, we will relate the blow up of the solution to the occurrence of the synchronization of neurons. Fully stochastic simulations of a finite size network of leaky integrate-and-fire neurons are performed to illustrate our theoretical results.     

Generating oscillatory bursts from a network of regular spiking neurons without inhibition

Avian nucleus isthmi pars parvocellularis (Ipc) neurons are reciprocally connected with the layer 10 (L10) neurons in the optic tectum and respond with oscillatory bursts to visual stimulation. Our in vitro experiments show that both neuron types respond with regular spiking to somatic current injection and that the feedforward and feedback synaptic connections are excitatory, but of different strength and time course. To elucidate mechanisms of oscillatory bursting in this network of regularly spiking neurons, we investigated an experimentally constrained model of coupled leaky integrate-and-fire neurons with spike-rate adaptation. The model reproduces the observed Ipc oscillatory bursting in response to simulated visual stimulation. A scan through the model parameter volume reveals that Ipc oscillatory burst generation can be caused by strong and brief feedforward synaptic conductance changes. The mechanism is sensitive to the parameter values of spike-rate adaptation. In conclusion, we show that a network of regular-spiking neurons with feedforward excitation and spike-rate adaptation can generate oscillatory bursting in response to a constant input.     

Influence of temporal correlation of synaptic input on the rate and variability of firing in neurons

The spike trains that transmit information between neurons are stochastic. We used the theory of random point processes and simulation methods to investigate the influence of temporal correlation of synaptic input current on firing statistics. The theory accounts for two sources for temporal correlation: synchrony between spikes in presynaptic input trains and the unitary synaptic current time course. Simulations show that slow temporal correlation of synaptic input leads to high variability in firing. In a leaky integrate-and-fire neuron model with spike afterhyperpolarization the theory accurately predicts the firing rate when the spike threshold is higher than two standard deviations of the membrane potential fluctuations. For lower thresholds the spike afterhyperpolarization reduces the firing rate below the theory's predicted level when the synaptic correlation decays rapidly. If the synaptic correlation decays slower than the spike afterhyperpolarization, spike bursts can occur during single broad peaks of input fluctuations, increasing the firing rate over the prediction. Spike bursts lead to a coefficient of variation for the interspike intervals that can exceed one, suggesting an explanation of high coefficient of variation for interspike intervals observed in vivo.     

Does inhibition balance excitation in neocortex?

The distribution of inhibitory and excitatory synapses on neocortical neurons is at odds with a simple view that cortical functioning can persist by maintaining a balance between inhibitory and excitatory drives. Pyramidal cells can potentially be shut down by very powerful proximal inhibitory synapses, despite these accounting for perhaps less than 1% of their total number of synaptic inputs. Interneurons in contrast are dominated by excitatory inputs. These may be powerful enough to effect an apparent depolarizing block at the soma. In this extreme case though, models suggest that action potentials are generated down the axon, and the cells behave like integrate-and-fire neurons. We discuss possible network implications of these modelling studies.