Coherency and connectivity in oscillating neural networks: linear partialization analysis

 This paper studies the relation between the functional synaptic connections between two artificial neural networks and the correlation of their spiking activities. The model neurons had realistic non-oscillatory dynamic properties and the networks showed oscillatory behavior as a result of their internal synaptic connectivity. We found that both excitation and inhibition cause phase locking of the oscillating activities. When the two networks excite each other the oscillations synchronize with zero phase lag, whereas mutual inhibition between the networks resulted in an anti-phase (half period phase difference) synchronization. Correlations between the activities of the two networks can also be caused by correlated external inputs driving the systems (common input). Our analysis shows that when the networks exhibit oscillatory behavior and the rate of the common input is smaller than a characteristic network oscillator frequency, the cross-correlation functions between the activities of two systems still carry information about the mutual synaptic connectivity. This information can be retrieved with linear partialization, removing the influence of the common input. We further explored the network responses to periodic external input. We found that when the input is of a frequency smaller than a certain threshold, the network responds with bursts at the same frequency as the input. Above the threshold, the network responds with a fraction of the input frequency. This frequency threshold, characterizing the oscillatory properties of the network, is also found to determine the limit to which linear partialization works. Received: 20 October 1995 / Accepted in revised form: 20 May 1996     

Acquiring new information in a neuronal network: from Hebb's concept to homeostatic plasticity

Synaptic plasticity is the cellular mechanism underlying the phenomena of learning and memory. Much of the research on synaptic plasticity is based on the postulate of Hebb (1949) who proposed that, when a neuron repeatedly takes part in the activation of another neuron, the efficacy of the connections between these neurons is increased. Plasticity has been extensively studied, and often demonstrated through the processes of LTP (Long Term Potentiation) and LTD (Long Term Depression), which represent an increase and a decrease of the efficacy of long-term synaptic transmission. This review summarizes current knowledge concerning the cellular mechanisms of LTP and LTD, whether at the level of excitatory synapses, which have been the most studied, or at the level of inhibitory synapses. However, if we consider neuronal networks rather than the individual synapses, the consequences of synaptic plasticity need to be considered on a large scale to determine if the activity of networks are changed or not. Homeostatic plasticity takes into account the mechanisms which control the efficacy of synaptic transmission for all the synaptic inputs of a neuron. Consequently, this new concept deals with the coordinated activity of excitatory and inhibitory networks afferent to a neuron which maintain a controlled level of excitability during the acquisition of new information related to the potentiation or to the depression of synaptic efficacy. We propose that the protocols of stimulation used to induce plasticity at the synaptic level set up a "homeostatic potentiation" or a "homeostatic depression" of excitation and inhibition at the level of the neuronal networks. The coordination between excitatory and inhibitory circuits allows the neuronal networks to preserve a level of stable activity, thus avoiding episodes of hyper- or hypo-activity during the learning and memory phases.     

The Influence of Synaptic Weight Distribution on Neuronal Population Dynamics

The manner in which different distributions of synaptic weights onto cortical neurons shape their spiking activity remains open. To characterize a homogeneous neuronal population, we use the master equation for generalized leaky integrate-and-fire neurons with shot-noise synapses. We develop fast semi-analytic numerical methods to solve this equation for either current or conductance synapses, with and without synaptic depression. We show that its solutions match simulations of equivalent neuronal networks better than those of the Fokker-Planck equation and we compute bounds on the network response to non-instantaneous synapses. We apply these methods to study different synaptic weight distributions in feed-forward networks. We characterize the synaptic amplitude distributions using a set of measures, called tail weight numbers, designed to quantify the preponderance of very strong synapses. Even if synaptic amplitude distributions are equated for both the total current and average synaptic weight, distributions with sparse but strong synapses produce higher responses for small inputs, leading to a larger operating range. Furthermore, despite their small number, such synapses enable the network to respond faster and with more stability in the face of external fluctuations.     

Impact of fast sodium channel inactivation on spike threshold dynamics and synaptic integration

Neurons spike when their membrane potential exceeds a threshold value. In central neurons, the spike threshold is not constant but depends on the stimulation. Thus, input-output properties of neurons depend both on the effect of presynaptic spikes on the membrane potential and on the dynamics of the spike threshold. Among the possible mechanisms that may modulate the threshold, one strong candidate is Na channel inactivation, because it specifically impacts spike initiation without affecting the membrane potential. We collected voltage-clamp data from the literature and we found, based on a theoretical criterion, that the properties of Na inactivation could indeed cause substantial threshold variability by itself. By analyzing simple neuron models with fast Na inactivation (one channel subtype), we found that the spike threshold is correlated with the mean membrane potential and negatively correlated with the preceding depolarization slope, consistent with experiments. We then analyzed the impact of threshold dynamics on synaptic integration. The difference between the postsynaptic potential (PSP) and the dynamic threshold in response to a presynaptic spike defines an effective PSP. When the neuron is sufficiently depolarized, this effective PSP is briefer than the PSP. This mechanism regulates the temporal window of synaptic integration in an adaptive way. Finally, we discuss the role of other potential mechanisms. Distal spike initiation, channel noise and Na activation dynamics cannot account for the observed negative slope-threshold relationship, while adaptive conductances (e.g. K+) and Na inactivation can. We conclude that Na inactivation is a metabolically efficient mechanism to control the temporal resolution of synaptic integration.     

Prediction and validation of a mechanism to control the threshold for inhibitory synaptic plasticity

Synaptic plasticity, neuronal activity‐dependent sustained alteration of the efficacy of synaptic transmission, underlies learning and memory. Activation of positive‐feedback signaling pathways by an increase in intracellular Ca²⁺ concentration ([Ca²⁺]_i) has been implicated in synaptic plasticity. However, the mechanism that determines the [Ca²⁺]_i threshold for inducing synaptic plasticity is elusive. Here, we developed a kinetic simulation model of inhibitory synaptic plasticity in the cerebellum, and systematically analyzed the behavior of intricate molecular networks composed of protein kinases, phosphatases, etc. The simulation showed that Ca²⁺/calmodulin‐dependent protein kinase II (CaMKII), which is essential for the induction of synaptic plasticity, was persistently activated or suppressed in response to different combinations of stimuli. The sustained CaMKII activation depended on synergistic actions of two positive‐feedback reactions, CaMKII autophosphorylation and CaMKII‐mediated inhibition of a CaM‐dependent phosphodiesterase, PDE1. The simulation predicted that PDE1‐mediated feedforward inhibition of CaMKII predominantly controls the Ca²⁺ threshold, which was confirmed by electrophysiological experiments in primary cerebellar cultures. Thus, combined application of simulation and experiments revealed that the Ca²⁺ threshold for the cerebellar inhibitory synaptic plasticity is primarily determined by PDE1.     

From synapse to network: models of information storage and retrieval in neural circuits

The mechanisms of information storage and retrieval in brain circuits are still the subject of debate. It is widely believed that information is stored at least in part through changes in synaptic connectivity in networks that encode this information and that these changes lead in turn to modifications of network dynamics, such that the stored information can be retrieved at a later time. Here, we review recent progress in deriving synaptic plasticity rules from experimental data and in understanding how plasticity rules affect the dynamics of recurrent networks. We show that the dynamics generated by such networks exhibit a large degree of diversity, depending on parameters, similar to experimental observations in vivo during delayed response tasks.     

Emergence of resonances in neural systems: the interplay between adaptive threshold and short-term synaptic plasticity

In this work we study the detection of weak stimuli by spiking (integrate-and-fire) neurons in the presence of certain level of noisy background neural activity. Our study has focused in the realistic assumption that the synapses in the network present activity-dependent processes, such as short-term synaptic depression and facilitation. Employing mean-field techniques as well as numerical simulations, we found that there are two possible noise levels which optimize signal transmission. This new finding is in contrast with the classical theory of stochastic resonance which is able to predict only one optimal level of noise. We found that the complex interplay between adaptive neuron threshold and activity-dependent synaptic mechanisms is responsible for this new phenomenology. Our main results are confirmed by employing a more realistic FitzHugh-Nagumo neuron model, which displays threshold variability, as well as by considering more realistic stochastic synaptic models and realistic signals such as poissonian spike trains.     

Information maintenance and statistical dependence reduction in simple neural networks

This study compares the ability of excitatory, feed-forward neural networks to construct good transformations on their inputs. The quality of such a transformation is judged by the minimization of two information measures: the information loss of the transformation and the statistical dependency of the output. The networks that are compared differ from each other in the parametric properties of their neurons and in their connectivity. The particular network parameters studied are output firing threshold, synaptic connectivity, and associative modification of connection weights. The network parameters that most directly affect firing levels are threshold and connectivity. Networks incorporating neurons with dynamic threshold adjustment produce better transformations. When firing threshold is optimized, sparser synaptic connectivity produces a better transformation than denser connectivity. Associative modification of synaptic weights confers only a slight advantage in the construction of optimal transformations. Additionally, our research shows that some environments are better suited than others for recoding. Specifically, input environments high in statistical dependence, i.e. those environments most in need of recoding, are more likely to undergo successful transformations.     

Successful Reconstruction of a Physiological Circuit with Known Connectivity from Spiking Activity Alone

Identifying the structure and dynamics of synaptic interactions between neurons is the first step to understanding neural network dynamics. The presence of synaptic connections is traditionally inferred through the use of targeted stimulation and paired recordings or by post-hoc histology. More recently, causal network inference algorithms have been proposed to deduce connectivity directly from electrophysiological signals, such as extracellularly recorded spiking activity. Usually, these algorithms have not been validated on a neurophysiological data set for which the actual circuitry is known. Recent work has shown that traditional network inference algorithms based on linear models typically fail to identify the correct coupling of a small central pattern generating circuit in the stomatogastric ganglion of the crab Cancer borealis. In this work, we show that point process models of observed spike trains can guide inference of relative connectivity estimates that match the known physiological connectivity of the central pattern generator up to a choice of threshold. We elucidate the necessary steps to derive faithful connectivity estimates from a model that incorporates the spike train nature of the data. We then apply the model to measure changes in the effective connectivity pattern in response to two pharmacological interventions, which affect both intrinsic neural dynamics and synaptic transmission. Our results provide the first successful application of a network inference algorithm to a circuit for which the actual physiological synapses between neurons are known. The point process methodology presented here generalizes well to larger networks and can describe the statistics of neural populations. In general we show that advanced statistical models allow for the characterization of effective network structure, deciphering underlying network dynamics and estimating information-processing capabilities.     

Impact of network topology on inference of synaptic connectivity from multi-neuronal spike data simulated by a large-scale cortical network model

Many mechanisms of neural processing rely critically upon the synaptic connectivity between neurons. As our ability to simultaneously record from large populations of neurons expands, the ability to infer network connectivity from this data has become a major goal of computational neuroscience. To address this issue, we employed several different methods to infer synaptic connections from simulated spike data from a realistic local cortical network model. This approach allowed us to directly compare the accuracy of different methods in predicting synaptic connectivity. We compared the performance of model-free (coherence measure and transfer entropy) and model-based (coupled escape rate model) methods of connectivity inference, applying those methods to the simulated spike data from the model networks with different network topologies. Our results indicate that the accuracy of the inferred connectivity was higher for highly clustered, near regular, or small-world networks, while accuracy was lower for random networks, irrespective of which analysis method was employed. Among the employed methods, the model-based method performed best. This model performed with higher accuracy, was less sensitive to threshold changes, and required less data to make an accurate assessment of connectivity. Given that cortical connectivity tends to be highly clustered, our results outline a powerful analytical tool for inferring local synaptic connectivity from observations of spontaneous activity.     

Synaptic clustering by dendritic signalling mechanisms

Dendritic signal integration is one of the fundamental building blocks of information processing in the brain. Dendrites are endowed with mechanisms of nonlinear summation of synaptic inputs leading to regenerative dendritic events including local sodium, NMDA and calcium spikes. The generation of these events requires distinct spatio-temporal activation patterns of synaptic inputs. We hypothesise that the recent findings on dendritic spikes and local synaptic plasticity rules suggest clustering of common inputs along a subregion of a dendritic branch. These clusters may enable dendrites to separately threshold groups of functionally similar inputs, thus allowing single neurons to act as a superposition of many separate integrate and fire units. Ultimately, these properties expand our understanding about the computational power of neuronal networks.     

The mechanisms for compression and reflection of cortical waves

Waves are common in cortical networks and may be important for carrying information about a stimulus from one local circuit to another. In a recent study of visually evoked waves in rat cortex, compression and reflection of waves are observed as the activation passes from visual areas V1 to V2. The authors of this study apply bicuculline (BMI) and demonstrate that the reflection disappears. They conclude that inhibition plays a major role in compression and reflection. We present several models for propagating waves in heterogeneous media and show that the velocity and thus compression depends weakly on inhibition. We propose that the main site of action of BMI with respect to wave propagation is on the threshold for firing which we suggest is related to action on potassium channels. We combine numerical and analytic methods to explore both compression and reflection in an excitable system with synaptic coupling.     

Are binary synapses superior to graded weight representations in stochastic attractor networks?

Synaptic plasticity is an underlying mechanism of learning and memory in neural systems, but it is controversial whether synaptic efficacy is modulated in a graded or binary manner. It has been argued that binary synaptic weights would be less susceptible to noise than graded weights, which has impelled some theoretical neuroscientists to shift from the use of graded to binary weights in their models. We compare retrieval performance of models using both binary and graded weight representations through numerical simulations of stochastic attractor networks. We also investigate stochastic attractor models using multiple discrete levels of weight states, and then investigate the optimal threshold for dilution of binary weight representations. Our results show that a binary weight representation is not less susceptible to noise than a graded weight representation in stochastic attractor models, and we find that the load capacities with an increasing number of weight states rapidly reach the load capacity with graded weights. The optimal threshold for dilution of binary weight representations under stochastic conditions occurs when approximately 50% of the smallest weights are set to zero.     

Mechanism of activation of human c-KIT kinase by internal tandem duplications of the juxtamembrane domain and point mutations at aspartic acid 816

The patients suffering from acidosis usually sign psychological deficits. The cerebral dysfunction is reportedly caused by an acid-induced functional impairment of GABAergic neurons; however, the role of pyramidal neurons in this process remains unclear. By using electrophysiological method and changing extracellular pH, we investigated the influence of acidic environment on pyramidal neurons in the cortical slices, such as their ability of firing spikes and response to synaptic inputs. A low pH of artificial cerebral spinal fluid elevates the responses of pyramidal neurons to excitatory synaptic inputs and their ability of encoding digital spikes, as well as reduces the signal transmission at GABAergic synapses. The elevated ability of neuronal spiking is associated with the decreases of refractory periods and threshold potentials. Therefore, acidosis deteriorates brain functions through making the activities between cortical pyramidal neurons and GABAergic neurons imbalanced toward the overexcitation of neural networks, a process similar to neural excitotoxicity.     

Frequency Control in Synchronized Networks of Inhibitory Neurons

We analyze the control of frequency for a synchronized inhibitory neuronal network. The analysis is done for a reduced membrane model with a biophysically based synaptic influence. We argue that such a reduced model can quantitatively capture the frequency behavior of a larger class of neuronal models. We show that in different parameter regimes, the network frequency depends in different ways on the intrinsic and synaptic time constants. Only in one portion of the parameter space, called phasic, is the network period proportional to the synaptic decay time. These results are discussed in connection with previous work of the authors, which showed that for mildly heterogeneous networks, the synchrony breaks down, but coherence is preserved much more for systems in the phasic regime than in the other regimes. These results imply that for mildly heterogeneous networks, the existence of a coherent rhythm implies a linear dependence of the network period on synaptic decay time and a much weaker dependence on the drive to the cells. We give experimental evidence for this conclusion.     

Self-Organization of Microcircuits in Networks of Spiking Neurons with Plastic Synapses

The synaptic connectivity of cortical networks features an overrepresentation of certain wiring motifs compared to simple random-network models. This structure is shaped, in part, by synaptic plasticity that promotes or suppresses connections between neurons depending on their joint spiking activity. Frequently, theoretical studies focus on how feedforward inputs drive plasticity to create this network structure. We study the complementary scenario of self-organized structure in a recurrent network, with spike timing-dependent plasticity driven by spontaneous dynamics. We develop a self-consistent theory for the evolution of network structure by combining fast spiking covariance with a slow evolution of synaptic weights. Through a finite-size expansion of network dynamics we obtain a low-dimensional set of nonlinear differential equations for the evolution of two-synapse connectivity motifs. With this theory in hand, we explore how the form of the plasticity rule drives the evolution of microcircuits in cortical networks. When potentiation and depression are in approximate balance, synaptic dynamics depend on weighted divergent, convergent, and chain motifs. For additive, Hebbian STDP these motif interactions create instabilities in synaptic dynamics that either promote or suppress the initial network structure. Our work provides a consistent theoretical framework for studying how spiking activity in recurrent networks interacts with synaptic plasticity to determine network structure.     

Efficient Transmission of Subthreshold Signals in Complex Networks of Spiking Neurons

We investigate the efficient transmission and processing of weak, subthreshold signals in a realistic neural medium in the presence of different levels of the underlying noise. Assuming Hebbian weights for maximal synaptic conductances—that naturally balances the network with excitatory and inhibitory synapses—and considering short-term synaptic plasticity affecting such conductances, we found different dynamic phases in the system. This includes a memory phase where population of neurons remain synchronized, an oscillatory phase where transitions between different synchronized populations of neurons appears and an asynchronous or noisy phase. When a weak stimulus input is applied to each neuron, increasing the level of noise in the medium we found an efficient transmission of such stimuli around the transition and critical points separating different phases for well-defined different levels of stochasticity in the system. We proved that this intriguing phenomenon is quite robust, as it occurs in different situations including several types of synaptic plasticity, different type and number of stored patterns and diverse network topologies, namely, diluted networks and complex topologies such as scale-free and small-world networks. We conclude that the robustness of the phenomenon in different realistic scenarios, including spiking neurons, short-term synaptic plasticity and complex networks topologies, make very likely that it could also occur in actual neural systems as recent psycho-physical experiments suggest.     

A general diffusion model for analyzing the efficacy of synaptic input to threshold neurons

We describe a general diffusion model for analyzing the efficacy of individual synaptic inputs to threshold neurons. A formal expression is obtained for the system propagator which, when given an arbitrary initial state for the cell, yields the conditional probability distribution for the state at all later times. The propagator for a cell with a finite threshold is written as a series expansion, such that each term in the series depends only on the infinite threshold propagator, which in the diffusion limit reduces to a Gaussian form. This procedure admits a graphical representation in terms of an infinite sequence of diagrams. To connect the theory to experiment, we construct an analytical expression for the primary correlation kernel (PCK) which profiles the change in the instantaneous firing rate produced by a single postsynaptic potential (PSP). Explicit solutions are obtained in the diffusion limit to first order in perturbation theory. Our approximate expression resembles the PCK obtained by computer simulation, with the accuracy depending strongly on the mode of firing. The theory is most accurate when the synaptic input drives the membrane potential to a mean level more than one standard deviation below the firing threshold, making such cells highly sensitive to synchronous synaptic input.     

Inhibitory Contribution to Suprathreshold Corticostriatal Responses: An Experimental and Modeling Study

Neostriatal neurons may undergo events of spontaneous synchronization as those observed in recurrent networks of excitatory neurons, even when cortical afferents are transected. It is necessary to explain these events because the neostriatum is a recurrent network of inhibitory neurons. Synchronization of neuronal activity may be caused by plateau-like depolarizations. Plateau-like orthodromic depolarizations that resemble up-states in medium spiny neostriatal neurons (MSNs) may be induced by a single corticostriatal suprathreshold stimulus. Slow synaptic depolarizations may last hundreds of milliseconds, decay slower than the monosynaptic glutamatergic synaptic potentials that induce them, and sustain repetitive firing. Because inhibitory inputs impinging onto MSNs have a reversal potential above the resting membrane potential but below the threshold for firing, they conform a type of “shunting inhibition”. This work asks if shunting GABAergic inputs onto MSNs arrive asynchronously enough as to help in sustaining the plateau-like corticostriatal response after a single cortical stimulus. This may help to begin explaining autonomous processing in the striatal micro-circuitry in the presence of a tonic excitatory drive and independently of spatio-temporally organized inputs. It is shown here that besides synaptic currents from AMPA/KA- and NMDA-receptors, as well as L-type intrinsic Ca²⁺- currents, inhibitory synapses help in maintaining the slow depolarization, although they accomplish the role of depressing firing at the beginning of the response. We then used a NEURON model of spiny cells to show that inhibitory synapses arriving asynchronously on the dendrites can help to simulate a plateau potential similar to that observed experimentally. Electronic supplementary material  The online version of this article (doi:) contains supplementary material, which is available to authorized users.     

A cooperation and competition based simple cell receptive field model and study of feed-forward linear and nonlinear contributions to orientation selectivity

We present a model for development of orientation selectivity in layer IV simple cells. Receptive field (RF) development in the model, is determined by diffusive cooperation and resource limited competition guided axonal growth and retraction in geniculocortical pathway. The simulated cortical RFs resemble experimental RFs. The receptive field model is incorporated in a three-layer visual pathway model consisting of retina, LGN and cortex. We have studied the effect of activity dependent synaptic scaling on orientation tuning of cortical cells. The mean value of hwhh (half width at half the height of maximum response) in simulated cortical cells is 58° when we consider only the linear excitatory contribution from LGN. We observe a mean improvement of 22.8° in tuning response due to the non-linear spiking mechanisms that include effects of threshold voltage and synaptic scaling factor.