Oscillations and Synchrony in Large-scale Cortical Network Models

Intrinsic neuronal and circuit properties control the responses of large ensembles of neurons by creating spatiotemporal patterns of activity that are used for sensory processing, memory formation, and other cognitive tasks. The modeling of such systems requires computationally efficient single-neuron models capable of displaying realistic response properties. We developed a set of reduced models based on difference equations (map-based models) to simulate the intrinsic dynamics of biological neurons. These phenomenological models were designed to capture the main response properties of specific types of neurons while ensuring realistic model behavior across a sufficient dynamic range of inputs. This approach allows for fast simulations and efficient parameter space analysis of networks containing hundreds of thousands of neurons of different types using a conventional workstation. Drawing on results obtained using large-scale networks of map-based neurons, we discuss spatiotemporal cortical network dynamics as a function of parameters that affect synaptic interactions and intrinsic states of the neurons.     

Minimal Hodgkin–Huxley type models for different classes of cortical and thalamic neurons

We review here the development of Hodgkin–Huxley (HH) type models of cerebral cortex and thalamic neurons for network simulations. The intrinsic electrophysiological properties of cortical neurons were analyzed from several preparations, and we selected the four most prominent electrophysiological classes of neurons. These four classes are “fast spiking” “regular spiking” “intrinsically bursting” and “low-threshold spike” cells. For each class, we fit “minimal” HH type models to experimental data. The models contain the minimal set of voltage-dependent currents to account for the data. To obtain models as generic as possible, we used data from different preparations in vivo and in vitro, such as rat somatosensory cortex and thalamus, guinea-pig visual and frontal cortex, ferret visual cortex, cat visual cortex and cat association cortex. For two cell classes, we used automatic fitting procedures applied to several cells, which revealed substantial cell-to-cell variability within each class. The selection of such cellular models constitutes a necessary step towards building network simulations of the thalamocortical system with realistic cellular dynamical properties.     

Firing patterns in the adaptive exponential integrate-and-fire model

For simulations of large spiking neuron networks, an accurate, simple and versatile single-neuron modeling framework is required. Here we explore the versatility of a simple two-equation model: the adaptive exponential integrate-and-fire neuron. We show that this model generates multiple firing patterns depending on the choice of parameter values, and present a phase diagram describing the transition from one firing type to another. We give an analytical criterion to distinguish between continuous adaption, initial bursting, regular bursting and two types of tonic spiking. Also, we report that the deterministic model is capable of producing irregular spiking when stimulated with constant current, indicating low-dimensional chaos. Lastly, the simple model is fitted to real experiments of cortical neurons under step current stimulation. The results provide support for the suitability of simple models such as the adaptive exponential integrate-and-fire neuron for large network simulations.     

Dynamics of spontaneous activity in random networks with multiple neuron subtypes and synaptic noise

During slow-wave sleep, brain electrical activity is dominated by the slow (< 1 Hz) electroencephalogram (EEG) oscillations characterized by the periodic transitions between active (or Up) and silent (or Down) states in the membrane voltage of the cortical and thalamic neurons. Sleep slow oscillation is believed to play critical role in consolidation of recent memories. Past computational studies, based on the Hodgkin-Huxley type neuronal models, revealed possible intracellular and network mechanisms of the neuronal activity during sleep, however, they failed to explore the large-scale cortical network dynamics depending on collective behavior in the large populations of neurons. In this new study, we developed a novel class of reduced discrete time spiking neuron models for large-scale network simulations of wake and sleep dynamics. In addition to the spiking mechanism, the new model implemented nonlinearities capturing effects of the leak current, the Ca²⁺ dependent K⁺ current and the persistent Na⁺ current that were found to be critical for transitions between Up and Down states of the slow oscillation. We applied the new model to study large-scale two-dimensional cortical network activity during slow-wave sleep. Our study explained traveling wave dynamics and characteristic synchronization properties of transitions between Up and Down states of the slow oscillation as observed in vivo in recordings from cats. We further predict a critical role of synaptic noise and slow adaptive currents for spike sequence replay as found during sleep related memory consolidation.     

Experimental analysis and computational modeling of interburst intervals in spontaneous activity of cortical neuronal culture

Rhythmic bursting is the most striking behavior of cultured cortical networks and may start in the second week after plating. In this study, we focus on the intervals between spontaneously occurring bursts, and compare experimentally recorded values with model simulations. In the models, we use standard neurons and synapses, with physiologically plausible parameters taken from literature. All networks had a random recurrent architecture with sparsely connected neurons. The number of neurons varied between 500 and 5,000. We find that network models with homogeneous synaptic strengths produce asynchronous spiking or stable regular bursts. The latter, however, are in a range not seen in recordings. By increasing the synaptic strength in a (randomly chosen) subset of neurons, our simulations show interburst intervals (IBIs) that agree better with in vitro experiments. In this regime, called weakly synchronized, the models produce irregular network bursts, which are initiated by neurons with relatively stronger synapses. In some noise-driven networks, a subthreshold, deterministic, input is applied to neurons with strong synapses, to mimic pacemaker network drive. We show that models with such "intrinsically active neurons" (pacemaker-driven models) tend to generate IBIs that are determined by the frequency of the fastest pacemaker and do not resemble experimental data. Alternatively, noise-driven models yield realistic IBIs. Generally, we found that large-scale noise-driven neuronal network models required synaptic strengths with a bimodal distribution to reproduce the experimentally observed IBI range. Our results imply that the results obtained from small network models cannot simply be extrapolated to models of more realistic size. Synaptic strengths in large-scale neuronal network simulations need readjustment to a bimodal distribution, whereas small networks do not require such changes.     

A Unified Approach to Linking Experimental,Statistical and Computational Analysis of Spike Train Data

A fundamental issue in neuroscience is how to identify the multiple biophysical mechanisms through which neurons generate observed patterns of spiking activity. In previous work, we proposed a method for linking observed patterns of spiking activity to specific biophysical mechanisms based on a state space modeling framework and a sequential Monte Carlo, or particle filter, estimation algorithm. We have shown, in simulation, that this approach is able to identify a space of simple biophysical models that were consistent with observed spiking data (and included the model that generated the data), but have yet to demonstrate the application of the method to identify realistic currents from real spike train data. Here, we apply the particle filter to spiking data recorded from rat layer V cortical neurons, and correctly identify the dynamics of an slow, intrinsic current. The underlying intrinsic current is successfully identified in four distinct neurons, even though the cells exhibit two distinct classes of spiking activity: regular spiking and bursting. This approach – linking statistical, computational, and experimental neuroscience – provides an effective technique to constrain detailed biophysical models to specific mechanisms consistent with observed spike train data.     

Cellular Adaptation Facilitates Sparse and Reliable Coding in Sensory Pathways

Most neurons in peripheral sensory pathways initially respond vigorously when a preferred stimulus is presented, but adapt as stimulation continues. It is unclear how this phenomenon affects stimulus coding in the later stages of sensory processing. Here, we show that a temporally sparse and reliable stimulus representation develops naturally in sequential stages of a sensory network with adapting neurons. As a modeling framework we employ a mean-field approach together with an adaptive population density treatment, accompanied by numerical simulations of spiking neural networks. We find that cellular adaptation plays a critical role in the dynamic reduction of the trial-by-trial variability of cortical spike responses by transiently suppressing self-generated fast fluctuations in the cortical balanced network. This provides an explanation for a widespread cortical phenomenon by a simple mechanism. We further show that in the insect olfactory system cellular adaptation is sufficient to explain the emergence of the temporally sparse and reliable stimulus representation in the mushroom body. Our results reveal a generic, biophysically plausible mechanism that can explain the emergence of a temporally sparse and reliable stimulus representation within a sequential processing architecture.     

A case for spiking neural network simulation based on configurable multiple-FPGA systems

Recent neuropsychological research has begun to reveal that neurons encode information in the timing of spikes. Spiking neural network simulations are a flexible and powerful method for investigating the behaviour of neuronal systems. Simulation of the spiking neural networks in software is unable to rapidly generate output spikes in large-scale of neural network. An alternative approach, hardware implementation of such system, provides the possibility to generate independent spikes precisely and simultaneously output spike waves in real time, under the premise that spiking neural network can take full advantage of hardware inherent parallelism. We introduce a configurable FPGA-oriented hardware platform for spiking neural network simulation in this work. We aim to use this platform to combine the speed of dedicated hardware with the programmability of software so that it might allow neuroscientists to put together sophisticated computation experiments of their own model. A feed-forward hierarchy network is developed as a case study to describe the operation of biological neural systems (such as orientation selectivity of visual cortex) and computational models of such systems. This model demonstrates how a feed-forward neural network constructs the circuitry required for orientation selectivity and provides platform for reaching a deeper understanding of the primate visual system. In the future, larger scale models based on this framework can be used to replicate the actual architecture in visual cortex, leading to more detailed predictions and insights into visual perception phenomenon.     

State transitions through inhibitory interneurons in a cortical network model

Inhibitory interneurons shape the spiking characteristics and computational properties of cortical networks. Interneuron subtypes can precisely regulate cortical function but the roles of interneuron subtypes for promoting different regimes of cortical activity remains unclear. Therefore, we investigated the impact of fast spiking and non-fast spiking interneuron subtypes on cortical activity using a network model with connectivity and synaptic properties constrained by experimental data. We found that network properties were more sensitive to modulation of the fast spiking population, with reductions of fast spiking excitability generating strong spike correlations and network oscillations. Paradoxically, reduced fast spiking excitability produced a reduction of global excitation-inhibition balance and features of an inhibition stabilised network, in which firing rates were driven by the activity of excitatory neurons within the network. Further analysis revealed that the synaptic interactions and biophysical features associated with fast spiking interneurons, in particular their rapid intrinsic response properties and short synaptic latency, enabled this state transition by enhancing gain within the excitatory population. Therefore, fast spiking interneurons may be uniquely positioned to control the strength of recurrent excitatory connectivity and the transition to an inhibition stabilised regime. Overall, our results suggest that interneuron subtypes can exert selective control over excitatory gain allowing for differential modulation of global network state.     

Stochastic Computations in Cortical Microcircuit Models

Experimental data from neuroscience suggest that a substantial amount of knowledge is stored in the brain in the form of probability distributions over network states and trajectories of network states. We provide a theoretical foundation for this hypothesis by showing that even very detailed models for cortical microcircuits, with data-based diverse nonlinear neurons and synapses, have a stationary distribution of network states and trajectories of network states to which they converge exponentially fast from any initial state. We demonstrate that this convergence holds in spite of the non-reversibility of the stochastic dynamics of cortical microcircuits. We further show that, in the presence of background network oscillations, separate stationary distributions emerge for different phases of the oscillation, in accordance with experimentally reported phase-specific codes. We complement these theoretical results by computer simulations that investigate resulting computation times for typical probabilistic inference tasks on these internally stored distributions, such as marginalization or marginal maximum-a-posteriori estimation. Furthermore, we show that the inherent stochastic dynamics of generic cortical microcircuits enables them to quickly generate approximate solutions to difficult constraint satisfaction problems, where stored knowledge and current inputs jointly constrain possible solutions. This provides a powerful new computing paradigm for networks of spiking neurons, that also throws new light on how networks of neurons in the brain could carry out complex computational tasks such as prediction, imagination, memory recall and problem solving.     

Probabilistic inference in general graphical models through sampling in stochastic networks of spiking neurons

An important open problem of computational neuroscience is the generic organization of computations in networks of neurons in the brain. We show here through rigorous theoretical analysis that inherent stochastic features of spiking neurons, in combination with simple nonlinear computational operations in specific network motifs and dendritic arbors, enable networks of spiking neurons to carry out probabilistic inference through sampling in general graphical models. In particular, it enables them to carry out probabilistic inference in Bayesian networks with converging arrows ("explaining away") and with undirected loops, that occur in many real-world tasks. Ubiquitous stochastic features of networks of spiking neurons, such as trial-to-trial variability and spontaneous activity, are necessary ingredients of the underlying computational organization. We demonstrate through computer simulations that this approach can be scaled up to neural emulations of probabilistic inference in fairly large graphical models, yielding some of the most complex computations that have been carried out so far in networks of spiking neurons.     

Computational Modeling of Seizure Dynamics Using Coupled Neuronal Networks: Factors Shaping Epileptiform Activity

Epileptic seizure dynamics span multiple scales in space and time. Understanding seizure mechanisms requires identifying the relations between seizure components within and across these scales, together with the analysis of their dynamical repertoire. Mathematical models have been developed to reproduce seizure dynamics across scales ranging from the single neuron to the neural population. In this study, we develop a network model of spiking neurons and systematically investigate the conditions, under which the network displays the emergent dynamic behaviors known from the Epileptor, which is a well-investigated abstract model of epileptic neural activity. This approach allows us to study the biophysical parameters and variables leading to epileptiform discharges at cellular and network levels. Our network model is composed of two neuronal populations, characterized by fast excitatory bursting neurons and regular spiking inhibitory neurons, embedded in a common extracellular environment represented by a slow variable. By systematically analyzing the parameter landscape offered by the simulation framework, we reproduce typical sequences of neural activity observed during status epilepticus. We find that exogenous fluctuations from extracellular environment and electro-tonic couplings play a major role in the progression of the seizure, which supports previous studies and further validates our model. We also investigate the influence of chemical synaptic coupling in the generation of spontaneous seizure-like events. Our results argue towards a temporal shift of typical spike waves with fast discharges as synaptic strengths are varied. We demonstrate that spike waves, including interictal spikes, are generated primarily by inhibitory neurons, whereas fast discharges during the wave part are due to excitatory neurons. Simulated traces are compared with in vivo experimental data from rodents at different stages of the disorder. We draw the conclusion that slow variations of global excitability, due to exogenous fluctuations from extracellular environment, and gap junction communication push the system into paroxysmal regimes. We discuss potential mechanisms underlying such machinery and the relevance of our approach, supporting previous detailed modeling studies and reflecting on the limitations of our methodology.     

Neural dynamics as sampling: a model for stochastic computation in recurrent networks of spiking neurons

The organization of computations in networks of spiking neurons in the brain is still largely unknown, in particular in view of the inherently stochastic features of their firing activity and the experimentally observed trial-to-trial variability of neural systems in the brain. In principle there exists a powerful computational framework for stochastic computations, probabilistic inference by sampling, which can explain a large number of macroscopic experimental data in neuroscience and cognitive science. But it has turned out to be surprisingly difficult to create a link between these abstract models for stochastic computations and more detailed models of the dynamics of networks of spiking neurons. Here we create such a link and show that under some conditions the stochastic firing activity of networks of spiking neurons can be interpreted as probabilistic inference via Markov chain Monte Carlo (MCMC) sampling. Since common methods for MCMC sampling in distributed systems, such as Gibbs sampling, are inconsistent with the dynamics of spiking neurons, we introduce a different approach based on non-reversible Markov chains that is able to reflect inherent temporal processes of spiking neuronal activity through a suitable choice of random variables. We propose a neural network model and show by a rigorous theoretical analysis that its neural activity implements MCMC sampling of a given distribution, both for the case of discrete and continuous time. This provides a step towards closing the gap between abstract functional models of cortical computation and more detailed models of networks of spiking neurons.     

Spiking behavior and epileptiform oscillations in a discrete model of cortical neural networks

Summary We investigate the phenomenon of epileptiform activity using a discrete model of cortical neural networks. Our model is reduced to the elementary features of neurons and assumes simplified dynamics of action potentials and postsynaptic potentials. The discrete model provides a comparably high simulation speed which allows the rendering of phase diagrams and simulations of large neural networks in reasonable time. Further the reduction to the basic features of neurons provides insight into the essentials of a possible mechanism of epilepsy. Our computer simulations suggest that the detailed dynamics of postsynaptic and action potentials are not indispensable for obtaining epileptiform behavior on the system level. The simulation results of autonomously evolving networks exhibit a regime in which the network dynamics spontaneously switch between fluctuating and oscillating behavior and produce isolated network spikes without external stimulation. Inhibitory neurons have been found to play an important part in the synchronization of neural firing: an increased number of synapses established by inhibitory neurons onto other neurons induces a transition to the spiking regime. A decreased frequency accompanying the hypersynchronous population activity has only occurred with slow inhibitory postsynaptic potentials.     

Running as fast as it can: How spiking dynamics form object groupings in the laminar circuits of visual cortex

How spiking neurons cooperate to control behavioral processes is a fundamental problem in computational neuroscience. Such cooperative dynamics are required during visual perception when spatially distributed image fragments are grouped into emergent boundary contours. Perceptual grouping is a challenge for spiking cells because its properties of collinear facilitation and analog sensitivity occur in response to binary spikes with irregular timing across many interacting cells. Some models have demonstrated spiking dynamics in recurrent laminar neocortical circuits, but not how perceptual grouping occurs. Other models have analyzed the fast speed of certain percepts in terms of a single feedforward sweep of activity, but cannot explain other percepts, such as illusory contours, wherein perceptual ambiguity can take hundreds of milliseconds to resolve by integrating multiple spikes over time. The current model reconciles fast feedforward with slower feedback processing, and binary spikes with analog network-level properties, in a laminar cortical network of spiking cells whose emergent properties quantitatively simulate parametric data from neurophysiological experiments, including the formation of illusory contours; the structure of non-classical visual receptive fields; and self-synchronizing gamma oscillations. These laminar dynamics shed new light on how the brain resolves local informational ambiguities through the use of properly designed nonlinear feedback spiking networks which run as fast as they can, given the amount of uncertainty in the data that they process.     

Propagation and synchronization of reverberatory bursts in developing cultured networks

Developing networks of neural systems can exhibit spontaneous, synchronous activities called neural bursts, which can be important in the organization of functional neural circuits. Before the network matures, the activity level of a burst can reverberate in repeated rise-and-falls in periods of hundreds of milliseconds following an initial wave-like propagation of spiking activity, while the burst itself lasts for seconds. To investigate the spatiotemporal structure of the reverberatory bursts, we culture dissociated, rat cortical neurons on a high-density multi-electrode array to record the dynamics of neural activity over the growth and maturation of the network. We find the synchrony of the spiking significantly reduced following the initial wave and the activities become broadly distributed spatially. The synchrony recovers as the system reverberates until the end of the burst. Using a propagation model we infer the spreading speed of the spiking activity, which increases as the culture ages. We perform computer simulations of the system using a physiological model of spiking networks in two spatial dimensions and find the parameters that reproduce the observed resynchronization of spiking in the bursts. An analysis of the simulated dynamics suggests that the depletion of synaptic resources causes the resynchronization. The spatial propagation dynamics of the simulations match well with observations over the course of a burst and point to an interplay of the synaptic efficacy and the noisy neural self-activation in producing the morphology of the bursts.     

Modeling Higher-Order Correlations within Cortical Microcolumns

We statistically characterize the population spiking activity obtained from simultaneous recordings of neurons across all layers of a cortical microcolumn. Three types of models are compared: an Ising model which captures pairwise correlations between units, a Restricted Boltzmann Machine (RBM) which allows for modeling of higher-order correlations, and a semi-Restricted Boltzmann Machine which is a combination of Ising and RBM models. Model parameters were estimated in a fast and efficient manner using minimum probability flow, and log likelihoods were compared using annealed importance sampling. The higher-order models reveal localized activity patterns which reflect the laminar organization of neurons within a cortical column. The higher-order models also outperformed the Ising model in log-likelihood: On populations of 20 cells, the RBM had 10% higher log-likelihood (relative to an independent model) than a pairwise model, increasing to 45% gain in a larger network with 100 spatiotemporal elements, consisting of 10 neurons over 10 time steps. We further removed the need to model stimulus-induced correlations by incorporating a peri-stimulus time histogram term, in which case the higher order models continued to perform best. These results demonstrate the importance of higher-order interactions to describe the structure of correlated activity in cortical networks. Boltzmann Machines with hidden units provide a succinct and effective way to capture these dependencies without increasing the difficulty of model estimation and evaluation.     

Integrate-and-fire vs Poisson models of LGN input to V1 cortex: noisier inputs reduce orientation selectivity

One of the reasons the visual cortex has attracted the interest of computational neuroscience is that it has well-defined inputs. The lateral geniculate nucleus (LGN) of the thalamus is the source of visual signals to the primary visual cortex (V1). Most large-scale cortical network models approximate the spike trains of LGN neurons as simple Poisson point processes. However, many studies have shown that neurons in the early visual pathway are capable of spiking with high temporal precision and their discharges are not Poisson-like. To gain an understanding of how response variability in the LGN influences the behavior of V1, we study response properties of model V1 neurons that receive purely feedforward inputs from LGN cells modeled either as noisy leaky integrate-and-fire (NLIF) neurons or as inhomogeneous Poisson processes. We first demonstrate that the NLIF model is capable of reproducing many experimentally observed statistical properties of LGN neurons. Then we show that a V1 model in which the LGN input to a V1 neuron is modeled as a group of NLIF neurons produces higher orientation selectivity than the one with Poisson LGN input. The second result implies that statistical characteristics of LGN spike trains are important for V1’s function. We conclude that physiologically motivated models of V1 need to include more realistic LGN spike trains that are less noisy than inhomogeneous Poisson processes.     

Analog VLSI implementation of resonate-and-fire neuron

We propose an analog integrated circuit that implements a resonate-and-fire neuron (RFN) model based on the Lotka-Volterra (LV) system. The RFN model is a spiking neuron model that has second-order membrane dynamics, and thus exhibits fast damped subthreshold oscillation, resulting in the coincidence detection, frequency preference, and post-inhibitory rebound. The RFN circuit has been derived from the LV system to mimic such dynamical behavior of the RFN model. Through circuit simulations, we demonstrate that the RFN circuit can act as a coincidence detector and a band-pass filter at circuit level even in the presence of additive white noise and background random activity. These results show that our circuit is expected to be useful for very large-scale integration (VLSI) implementation of functional spiking neural networks.     

Optimizing interneuron circuits for compartment-specific feedback inhibition

Cortical circuits process information by rich recurrent interactions between excitatory neurons and inhibitory interneurons. One of the prime functions of interneurons is to stabilize the circuit by feedback inhibition, but the level of specificity on which inhibitory feedback operates is not fully resolved. We hypothesized that inhibitory circuits could enable separate feedback control loops for different synaptic input streams, by means of specific feedback inhibition to different neuronal compartments. To investigate this hypothesis, we adopted an optimization approach. Leveraging recent advances in training spiking network models, we optimized the connectivity and short-term plasticity of interneuron circuits for compartment-specific feedback inhibition onto pyramidal neurons. Over the course of the optimization, the interneurons diversified into two classes that resembled parvalbumin (PV) and somatostatin (SST) expressing interneurons. Using simulations and mathematical analyses, we show that the resulting circuit can be understood as a neural decoder that inverts the nonlinear biophysical computations performed within the pyramidal cells. Our model provides a proof of concept for studying structure-function relations in cortical circuits by a combination of gradient-based optimization and biologically plausible phenomenological models.