Adaptive and phase selective spike timing dependent plasticity in synaptically coupled neuronal oscillators

We consider and analyze the influence of spike-timing dependent plasticity (STDP) on homeostatic states in synaptically coupled neuronal oscillators. In contrast to conventional models of STDP in which spike-timing affects weights of synaptic connections, we consider a model of STDP in which the time lags between pre- and/or post-synaptic spikes change internal state of pre- and/or post-synaptic neurons respectively. The analysis reveals that STDP processes of this type, modeled by a single ordinary differential equation, may ensure efficient, yet coarse, phase-locking of spikes in the system to a given reference phase. Precision of the phase locking, i.e. the amplitude of relative phase deviations from the reference, depends on the values of natural frequencies of oscillators and, additionally, on parameters of the STDP law. These deviations can be optimized by appropriate tuning of gains (i.e. sensitivity to spike-timing mismatches) of the STDP mechanism. However, as we demonstrate, such deviations can not be made arbitrarily small neither by mere tuning of STDP gains nor by adjusting synaptic weights. Thus if accurate phase-locking in the system is required then an additional tuning mechanism is generally needed. We found that adding a very simple adaptation dynamics in the form of slow fluctuations of the base line in the STDP mechanism enables accurate phase tuning in the system with arbitrary high precision. Adaptation operating at a slow time scale may be associated with extracellular matter such as matrix and glia. Thus the findings may suggest a possible role of the latter in regulating synaptic transmission in neuronal circuits.     

Unsupervised learning of visual features through spike timing dependent plasticity

Spike timing dependent plasticity (STDP) is a learning rule that modifies synaptic strength as a function of the relative timing of pre- and postsynaptic spikes. When a neuron is repeatedly presented with similar inputs, STDP is known to have the effect of concentrating high synaptic weights on afferents that systematically fire early, while postsynaptic spike latencies decrease. Here we use this learning rule in an asynchronous feedforward spiking neural network that mimics the ventral visual pathway and shows that when the network is presented with natural images, selectivity to intermediate-complexity visual features emerges. Those features, which correspond to prototypical patterns that are both salient and consistently present in the images, are highly informative and enable robust object recognition, as demonstrated on various classification tasks. Taken together, these results show that temporal codes may be a key to understanding the phenomenal processing speed achieved by the visual system and that STDP can lead to fast and selective responses.     

An implementation of reinforcement learning based on spike timing dependent plasticity

An explanatory model is developed to show how synaptic learning mechanisms modeled through spike-timing dependent plasticity (STDP) can result in long-term adaptations consistent with reinforcement learning models. In particular, the reinforcement learning model known as temporal difference (TD) learning has been used to model neuronal behavior in the orbitofrontal cortex (OFC) and ventral tegmental area (VTA) of macaque monkey during reinforcement learning. While some research has observed, empirically, a connection between STDP and TD, there has not been an explanatory model directly connecting TD to STDP. Through analysis of the learning dynamics that results from a general form of a STDP learning rule, the connection between STDP and TD is explained. We further demonstrate that a STDP learning rule drives the spike probability of a reward predicting neuronal population to a stable equilibrium. The equilibrium solution has an increasing slope where the steepness of the slope predicts the probability of the reward, similar to the results from electrophysiological recordings suggesting a different slope that predicts the value of the anticipated reward of Montague and Berns [Neuron 36(2):265–284, 2002]. This connection begins to shed light into more recent data gathered from VTA and OFC which are not well modeled by TD. We suggest that STDP provides the underlying mechanism for explaining reinforcement learning and other higher level perceptual and cognitive function. This material is based upon work supported by the National Science Foundation under Grants No. IOB-0445648 (PDR) and DMS-0408334 (GL) and by a Career Support grant from Portland State University (GL).     

Spike timing dependent plasticity finds the start of repeating patterns in continuous spike trains

Experimental studies have observed Long Term synaptic Potentiation (LTP) when a presynaptic neuron fires shortly before a postsynaptic neuron, and Long Term Depression (LTD) when the presynaptic neuron fires shortly after, a phenomenon known as Spike Timing Dependent Plasticity (STDP). When a neuron is presented successively with discrete volleys of input spikes STDP has been shown to learn 'early spike patterns', that is to concentrate synaptic weights on afferents that consistently fire early, with the result that the postsynaptic spike latency decreases, until it reaches a minimal and stable value. Here, we show that these results still stand in a continuous regime where afferents fire continuously with a constant population rate. As such, STDP is able to solve a very difficult computational problem: to localize a repeating spatio-temporal spike pattern embedded in equally dense 'distractor' spike trains. STDP thus enables some form of temporal coding, even in the absence of an explicit time reference. Given that the mechanism exposed here is simple and cheap it is hard to believe that the brain did not evolve to use it.     

Spike timing dependent plasticity promotes synchrony of inhibitory networks in the presence of heterogeneity

Recently Haas et al. (J Neurophysiol 96: 3305–3313, 2006), observed a novel form of spike timing dependent plasticity (iSTDP) in GABAergic synaptic couplings in layer II of the entorhinal cortex. Depending on the relative timings of the presynaptic input at time t _pre and the postsynaptic excitation at time t _post, the synapse is strengthened (Δt = t _post − t _pre > 0) or weakened (Δt < 0). The temporal dynamic range of the observed STDP rule was found to lie in the higher gamma frequency band (≥40 Hz), a frequency range important for several vital neuronal tasks. In this paper we study the function of this novel form of iSTDP in the synchronization of the inhibitory neuronal network. In particular we consider a network of two unidirectionally coupled interneurons (UCI) and two mutually coupled interneurons (MCI), in the presence of heterogeneity in the intrinsic firing rates of each coupled neuron. Using the method of spike time response curve (STRC), we show how iSTDP influences the dynamics of the coupled neurons, such that the pair synchronizes under moderately large heterogeneity in the firing rates. Using the general properties of the STRC for a Type-1 neuron model (Ermentrout, Neural Comput 8:979–1001, 1996) and the observed iSTDP we determine conditions on the initial configuration of the UCI network that would result in 1:1 in-phase synchrony between the two coupled neurons. We then demonstrate a similar enhancement of synchrony in the MCI with dynamic synaptic modulation. For the MCI we also consider heterogeneity introduced in the network through the synaptic parameters: the synaptic decay time of mutual inhibition and the self inhibition synaptic strength. We show that the MCI exhibits enhanced synchrony in the presence of all the above mentioned sources of heterogeneity and the mechanism for this enhanced synchrony is similar to the case of the UCI.     

On bifurcations and chaos in random neural networks

Chaos in nervous system is a fascinating but controversial field of investigation. To approach the role of chaos in the real brain, we theoretically and numerically investigate the occurrence of chaos inartificial neural networks. Most of the time, recurrent networks (with feedbacks) are fully connected. This architecture being not biologically plausible, the occurrence of chaos is studied here for a randomly diluted architecture. By normalizing the variance of synaptic weights, we produce a bifurcation parameter, dependent on this variance and on the slope of the transfer function, that allows a sustained activity and the occurrence of chaos when reaching a critical value. Even for weak connectivity and small size, we find numerical results in accordance with the theoretical ones previously established for fully connected infinite sized networks. The route towards chaos is numerically checked to be a quasi-periodic one, whatever the type of the first bifurcation is. Our results suggest that such high-dimensional networks behave like low-dimensional dynamical systems.     

Bursts shape the NMDA-R mediated spike timing dependent plasticity curve: role of burst interspike interval and GABAergic inhibition

Spike timing dependent plasticity (STDP) is a synaptic learning rule where the relative timing between the presynaptic and postsynaptic action potentials determines the sign and strength of synaptic plasticity. In its basic form STDP has an asymmetric form which incorporates both persistent increases and persistent decreases in synaptic strength. The basic form of STDP, however, is not a fixed property and depends on the dendritic location. An asymmetric curve is observed in the distal dendrites, whereas a symmetrical one is observed in the proximal ones. A recent computational study has shown that the transition from the asymmetry to symmetry is due to inhibition under certain conditions. Synapses have also been observed to be unreliable at generating plasticity when excitatory postsynaptic potentials and single spikes are paired at low frequencies. Bursts of spikes, however, are reliably signaled because transmitter release is facilitated. This article presents a two-compartment model of the CA1 pyramidal cell. The model is neurophysiologically plausible with its dynamics resulting from the interplay of many ionic and synaptic currents. Plasticity is measured by a deterministic Ca²⁺ dynamics model which measures the instantaneous calcium level and its time course in the dendrite and change the strength of the synapse accordingly. The model is validated to match the asymmetrical form of STDP from the pairing of a presynaptic (dendritic) and postsynaptic (somatic) spikes as observed experimentally. With the parameter set unchanged the model investigates how pairing of bursts with single spikes and bursts in the presence or absence of inhibition shapes the STDP curve. The model predicts that inhibition strength and frequency are not the only factors of the asymmetry-to-symmetry switch of the STDP curve. Burst interspike interval is another factor. This study is an important first step towards understanding how STDP is affected under natural firing patterns in vivo.     

Signal-to-signal neural networks for improved spike estimation from calcium imaging data

Spiking information of individual neurons is essential for functional and behavioral analysis in neuroscience research. Calcium imaging techniques are generally employed to obtain activities of neuronal populations. However, these techniques result in slowly-varying fluorescence signals with low temporal resolution. Estimating the temporal positions of the neuronal action potentials from these signals is a challenging problem. In the literature, several generative model-based and data-driven algorithms have been studied with varied levels of success. This article proposes a neural network-based signal-to-signal conversion approach, where it takes as input raw-fluorescence signal and learns to estimate the spike information in an end-to-end fashion. Theoretically, the proposed approach formulates the spike estimation as a single channel source separation problem with unknown mixing conditions. The source corresponding to the action potentials at a lower resolution is estimated at the output. Experimental studies on the spikefinder challenge dataset show that the proposed signal-to-signal conversion approach significantly outperforms state-of-the-art-methods in terms of Pearson’s correlation coefficient, Spearman’s rank correlation coefficient and yields comparable performance for the area under the receiver operating characteristics measure. We also show that the resulting system: (a) has low complexity with respect to existing supervised approaches and is reproducible; (b) is layer-wise interpretable, and (c) has the capability to generalize across different calcium indicators.     

Calcium dependent plasticity applied to repetitive transcranial magnetic stimulation with a neural field model

The calcium dependent plasticity (CaDP) approach to the modeling of synaptic weight change is applied using a neural field approach to realistic repetitive transcranial magnetic stimulation (rTMS) protocols. A spatially-symmetric nonlinear neural field model consisting of populations of excitatory and inhibitory neurons is used. The plasticity between excitatory cell populations is then evaluated using a CaDP approach that incorporates metaplasticity. The direction and size of the plasticity (potentiation or depression) depends on both the amplitude of stimulation and duration of the protocol. The breaks in the inhibitory theta-burst stimulation protocol are crucial to ensuring that the stimulation bursts are potentiating in nature. Tuning the parameters of a spike-timing dependent plasticity (STDP) window with a Monte Carlo approach to maximize agreement between STDP predictions and the CaDP results reproduces a realistically-shaped window with two regions of depression in agreement with the existing literature. Developing understanding of how TMS interacts with cells at a network level may be important for future investigation.     

Reinforcement learning with internal expectation in the random neural networks for cascaded decisions

The reinforcement learning scheme proposed in Halici (J. Biosystems 40 (1997) 83) for the random neural network (RNN) (Neural Computation 1 (1989) 502) is based on reward and performs well for stationary environments. However, when the environment is not stationary it suffers from getting stuck to the previously learned action and extinction is not possible. To overcome the problem, the reinforcement scheme is extended in Halici (Eur. J. Oper. Res., 126(2000) 288) by introducing a new weight update rule (E-rule) which takes into consideration the internal expectation of reinforcement. Although the E-rule is proposed for the RNN, it can be used for training learning automata or other intelligent systems based on reinforcement learning. This paper looks into the behavior of the learning scheme with internal expectation for the environments where the reinforcement is obtained after a sequence of cascaded decisions. The simulation results have shown that the RNN learns well and extinction is possible even for the cases with several decision steps and with hundreds of possible decision paths.     

On problem solving with Hopfield neural networks

Hopfield and Tank have shown that neural networks can be used to solve certain computationally hard problems, in particular they studied the Traveling Salesman Problem (TSP). Based on network simulation results they conclude that analog VLSI neural nets can be promising in solving these problems. Recently, Wilson and Pawley presented the results of their simulations which contradict the original results and cast doubts on the usefulness of neural nets. In this paper we give the results of our simulations that clarify some of the discrepancies. We also investigate the scaling of TSP solutions found by neural nets as the size of the problem increases. Further, we consider the neural net solution of the Clustering Problem, also a computationally hard problem, and discuss the types of problems that appear to be well suited for a neural net approach.     

Sequences of smoothly correlated patterns in neural networks with random transmission delays

A neural network with a broad distribution of transmission delays was used to study numerically the retrieval of sequences having several types of correlations between successive patterns. In the case of sequences consisting of patterns correlated for finite time, the quality of retrieval was found to be (more or less) independent of the pattern correlation width and the delay distribution. On the other hand the quality of retrieval is dependent on these factors in the case of sequences with pattern correlation functions with long time tails. Finally we have studied to what extent the storage capacity depends on the pattern correlation function and the delay distribution.     

Induction and propagation of transient synchronous activity in neural networks endowed with short-term plasticity

Transient, task related synchronous activity within neural populations has been recognized as the substrate of temporal coding in the brain. The mechanisms underlying inducing and propagation of transient synchronous activity are still unknown, and we propose that short-term plasticity (STP) of neural circuits may serve as a supplemental mechanism therein. By computational modeling, we showed that short-term facilitation greatly increases the reactivation rate of population spikes and decreases the latency of response to reactivation stimuli in local recurrent neural networks. Meanwhile, the timing of population spike reactivation is controlled by the memory effect of STP, and it is mediated primarily by the facilitation time constant. Furthermore, we demonstrated that synaptic facilitation dramatically enhances synchrony propagation in feedforward neural networks and that response timing mediated by synaptic facilitation offers a scheme for information routing. In addition, we verified that synaptic strengthening of intralayer or interlayer coupling enhances synchrony propagation, and we verified that other factors such as the delay of synaptic transmission and the mode of synaptic connectivity are also involved in regulating synchronous activity propagation. Overall, our results highlight the functional role of STP in regulating the inducing and propagation of transient synchronous activity, and they may inspire testable hypotheses for future experimental studies.     

The neurochip: promoting plasticity with a neural implant

A new discovery suggests that converting the brain's own natural activity into electrical stimuli that are delivered back into another brain region can induce long-term plastic change. This discovery could provide a powerful and useful addition to therapeutic uses of brain-machine interfaces.     

Stability and structural constraints of random brain networks with excitatory and inhibitory neural populations

The stability of brain networks with randomly connected excitatory and inhibitory neural populations is investigated using a simplified physiological model of brain electrical activity. Neural populations are randomly assigned to be excitatory or inhibitory and the stability of a brain network is determined by the spectrum of the network’s matrix of connection strengths. The probability that a network is stable is determined from its spectral density which is numerically determined and is approximated by a spectral distribution recently derived by Rajan and Abbott. The probability that a brain network is stable is maximum when the total connection strength into a population is approximately zero and is shown to depend on the arrangement of the excitatory and inhibitory connections and the parameters of the network. The maximum excitatory and inhibitory input into a structure allowed by stability occurs when the net input equals zero and, in contrast to networks with randomly distributed excitatory and inhibitory connections, substantially increases as the number of connections increases. Networks with the largest excitatory and inhibitory input allowed by stability have multiple marginally stable modes, are highly responsive and adaptable to external stimuli, have the same total input into each structure with minimal variance in the excitatory and inhibitory connection strengths, and have a wide range of flexible, adaptable, and complex behavior.     

Selecting neural networks for a committee decision

To improve recognition results, decisions of multiple neural networks can be aggregated into a committee decision. In contrast to the ordinary approach of utilizing all neural networks available to make a committee decision, we propose creating adaptive committees, which are specific for each input data point. A prediction network is used to identify classification neural networks to be fused for making a committee decision about a given input data point. The jth output value of the prediction network expresses the expectation level that the jth classification neural network will make a correct decision about the class label of a given input data point. The proposed technique is tested in three aggregation schemes, namely majority vote, averaging, and aggregation by the median rule and compared with the ordinary neural networks fusion approach. The effectiveness of the approach is demonstrated on two artificial and three real data sets.     

On the capacity of multilayer neural networks trained with backpropagation

The capacity of a layered neural network for learning hetero-associations is studied numerically as a function of the number M of hidden neurons. We find that there is a sharp change in the learning ability of the network as the number of hetero-associations increases. This fact allows us to define a maximum capacity C for a given architecture. It is found that C grows logarithmically with M.