Spike timing-dependent plasticity and memory

Spike timing-dependent plasticity (STDP) is a bidirectional form of synaptic plasticity discovered about 30 years ago and based on the relative timing of pre- and post-synaptic spiking activity with a millisecond precision. STDP is thought to be involved in the formation of memory but the millisecond-precision spike-timing required for STDP is difficult to reconcile with the much slower timescales of behavioral learning. This review therefore aims to expose and discuss recent findings about i) the multiple STDP learning rules at both excitatory and inhibitory synapses in vitro, ii) the contribution of STDP-like synaptic plasticity in the formation of memory in vivo and iii) the implementation of STDP rules in artificial neural networks and memristive devices.     

Spike timing-dependent plasticity of neural circuits

Recent findings of spike timing-dependent plasticity (STDP) have stimulated much interest among experimentalists and theorists. Beyond the traditional correlation-based Hebbian plasticity, STDP opens up new avenues for understanding information coding and circuit plasticity that depend on the precise timing of neuronal spikes. Here we summarize experimental characterization of STDP at various synapses, the underlying cellular mechanisms, and the associated changes in neuronal excitability and dendritic integration. We also describe STDP in the context of complex spike patterns and its dependence on the dendritic location of the synapse. Finally, we discuss timing-dependent modification of neuronal receptive fields and human visual perception and the computational significance of STDP as a synaptic learning rule.     

A Nonlinear Cable Framework for Bidirectional Synaptic Plasticity

Finding the rules underlying how axons of cortical neurons form neural circuits and modify their corresponding synaptic strength is the still subject of intense research. Experiments have shown that internal calcium concentration, and both the precise timing and temporal order of pre and postsynaptic action potentials, are important constituents governing whether the strength of a synapse located on the dendrite is increased or decreased. In particular, previous investigations focusing on spike timing-dependent plasticity (STDP) have typically observed an asymmetric temporal window governing changes in synaptic efficacy. Such a temporal window emphasizes that if a presynaptic spike, arriving at the synaptic terminal, precedes the generation of a postsynaptic action potential, then the synapse is potentiated; however if the temporal order is reversed, then depression occurs. Furthermore, recent experimental studies have now demonstrated that the temporal window also depends on the dendritic location of the synapse. Specifically, it was shown that in distal regions of the apical dendrite, the magnitude of potentiation was smaller and the window for depression was broader, when compared to observations from the proximal region of the dendrite. To date, the underlying mechanism(s) for such a distance-dependent effect is (are) currently unknown. Here, using the ionic cable theory framework in conjunction with the standard calcium based plasticity model, we show for the first time that such distance-dependent inhomogeneities in the temporal learning window for STDP can be largely explained by both the spatial and active properties of the dendrite.     

Intrinsic stability of temporally shifted spike-timing dependent plasticity

Spike-timing dependent plasticity (STDP), a widespread synaptic modification mechanism, is sensitive to correlations between presynaptic spike trains and it generates competition among synapses. However, STDP has an inherent instability because strong synapses are more likely to be strengthened than weak ones, causing them to grow in strength until some biophysical limit is reached. Through simulations and analytic calculations, we show that a small temporal shift in the STDP window that causes synchronous, or nearly synchronous, pre- and postsynaptic action potentials to induce long-term depression can stabilize synaptic strengths. Shifted STDP also stabilizes the postsynaptic firing rate and can implement both Hebbian and anti-Hebbian forms of competitive synaptic plasticity. Interestingly, the overall level of inhibition determines whether plasticity is Hebbian or anti-Hebbian. Even a random symmetric jitter of a few milliseconds in the STDP window can stabilize synaptic strengths while retaining these features. The same results hold for a shifted version of the more recent "triplet" model of STDP. Our results indicate that the detailed shape of the STDP window function near the transition from depression to potentiation is of the utmost importance in determining the consequences of STDP, suggesting that this region warrants further experimental study.     

The functional role of sequentially neuromodulated synaptic plasticity in behavioural learning

To survive, animals have to quickly modify their behaviour when the reward changes. The internal representations responsible for this are updated through synaptic weight changes, mediated by certain neuromodulators conveying feedback from the environment. In previous experiments, we discovered a form of hippocampal Spike-Timing-Dependent-Plasticity (STDP) that is sequentially modulated by acetylcholine and dopamine. Acetylcholine facilitates synaptic depression, while dopamine retroactively converts the depression into potentiation. When these experimental findings were implemented as a learning rule in a computational model, our simulations showed that cholinergic-facilitated depression is important for reversal learning. In the present study, we tested the model’s prediction by optogenetically inactivating cholinergic neurons in mice during a hippocampus-dependent spatial learning task with changing rewards. We found that reversal learning, but not initial place learning, was impaired, verifying our computational prediction that acetylcholine-modulated plasticity promotes the unlearning of old reward locations. Further, differences in neuromodulator concentrations in the model captured mouse-by-mouse performance variability in the optogenetic experiments. Our line of work sheds light on how neuromodulators enable the learning of new contingencies.     

LTD windows of the STDP learning rule and synaptic connections having a large transmission delay enable robust sequence learning amid background noise

Spike-timing-dependent synaptic plasticity (STDP) is a simple and effective learning rule for sequence learning. However, synapses being subject to STDP rules are readily influenced in noisy circumstances because synaptic conductances are modified by pre- and postsynaptic spikes elicited within a few tens of milliseconds, regardless of whether those spikes convey information or not. Noisy firing existing everywhere in the brain may induce irrelevant enhancement of synaptic connections through STDP rules and would result in uncertain memory encoding and obscure memory patterns. We will here show that the LTD windows of the STDP rules enable robust sequence learning amid background noise in cooperation with a large signal transmission delay between neurons and a theta rhythm, using a network model of the entorhinal cortex layer II with entorhinal-hippocampal loop connections. The important element of the present model for robust sequence learning amid background noise is the symmetric STDP rule having LTD windows on both sides of the LTP window, in addition to the loop connections having a large signal transmission delay and the theta rhythm pacing activities of stellate cells. Above all, the LTD window in the range of positive spike-timing is important to prevent influences of noise with the progress of sequence learning.     

A computational framework for cortical learning

Recent physiological findings have revealed that long-term adaptation of the synaptic strengths between cortical pyramidal neurons depends on the temporal order of presynaptic and postsynaptic spikes, which is called spike-timing-dependent plasticity (STDP) or temporally asymmetric Hebbian (TAH) learning. Here I prove by analytical means that a physiologically plausible variant of STDP adapts synaptic strengths such that the presynaptic spikes predict the postsynaptic spikes with minimal error. This prediction error model of STDP implies a mechanism for cortical memory: cortical tissue learns temporal spike patterns if these spike patterns are repeatedly elicited in a set of pyramidal neurons. The trained network finishes these patterns if their beginnings are presented, thereby recalling the memory. Implementations of the proposed algorithms may be useful for applications in voice recognition and computer vision.     

Mirrored STDP Implements Autoencoder Learning in a Network of Spiking Neurons

The autoencoder algorithm is a simple but powerful unsupervised method for training neural networks. Autoencoder networks can learn sparse distributed codes similar to those seen in cortical sensory areas such as visual area V1, but they can also be stacked to learn increasingly abstract representations. Several computational neuroscience models of sensory areas, including Olshausen & Field’s Sparse Coding algorithm, can be seen as autoencoder variants, and autoencoders have seen extensive use in the machine learning community. Despite their power and versatility, autoencoders have been difficult to implement in a biologically realistic fashion. The challenges include their need to calculate differences between two neuronal activities and their requirement for learning rules which lead to identical changes at feedforward and feedback connections. Here, we study a biologically realistic network of integrate-and-fire neurons with anatomical connectivity and synaptic plasticity that closely matches that observed in cortical sensory areas. Our choice of synaptic plasticity rules is inspired by recent experimental and theoretical results suggesting that learning at feedback connections may have a different form from learning at feedforward connections, and our results depend critically on this novel choice of plasticity rules. Specifically, we propose that plasticity rules at feedforward versus feedback connections are temporally opposed versions of spike-timing dependent plasticity (STDP), leading to a symmetric combined rule we call Mirrored STDP (mSTDP). We show that with mSTDP, our network follows a learning rule that approximately minimizes an autoencoder loss function. When trained with whitened natural image patches, the learned synaptic weights resemble the receptive fields seen in V1. Our results use realistic synaptic plasticity rules to show that the powerful autoencoder learning algorithm could be within the reach of real biological networks.     

Neuromodulation of STDP through short-term changes in firing causality

Spike timing dependent plasticity (STDP) likely plays an important role in forming and changing connectivity patterns between neurons in our brain. In a unidirectional synaptic connection between two neurons, it uses the causal relation between spiking activity of a presynaptic input neuron and a postsynaptic output neuron to change the strength of this connection. While the nature of STDP benefits unsupervised learning of correlated inputs, any incorporation of value into the learning process needs some form of reinforcement. Chemical neuromodulators such as Dopamine or Acetylcholine are thought to signal changes between external reward and internal expectation to many brain regions, including the basal ganglia. This effect is often modelled through a direct inclusion of the level of Dopamine as a third factor into the STDP rule. While this gives the benefit of direct control over synaptic modification, it does not account for observed instantaneous effects in neuronal activity on application of Dopamine agonists. Specifically, an instant facilitation of neuronal excitability in the striatum can not be explained by the only indirect effect that dopamine-modulated STDP has on a neuron’s firing pattern. We therefore propose a model for synaptic transmission where the level of neuromodulator does not directly influence synaptic plasticity, but instead alters the relative firing causality between pre- and postsynaptic neurons. Through the direct effect on postsynaptic activity, our rule allows indirect modulation of the learning outcome even with unmodulated, two-factor STDP. However, it also does not prohibit joint operation together with three-factor STDP rules.     

Phenomenological models of synaptic plasticity based on spike timing

Synaptic plasticity is considered to be the biological substrate of learning and memory. In this document we review phenomenological models of short-term and long-term synaptic plasticity, in particular spike-timing dependent plasticity (STDP). The aim of the document is to provide a framework for classifying and evaluating different models of plasticity. We focus on phenomenological synaptic models that are compatible with integrate-and-fire type neuron models where each neuron is described by a small number of variables. This implies that synaptic update rules for short-term or long-term plasticity can only depend on spike timing and, potentially, on membrane potential, as well as on the value of the synaptic weight, or on low-pass filtered (temporally averaged) versions of the above variables. We examine the ability of the models to account for experimental data and to fulfill expectations derived from theoretical considerations. We further discuss their relations to teacher-based rules (supervised learning) and reward-based rules (reinforcement learning). All models discussed in this paper are suitable for large-scale network simulations.     

The spike-timing dependence of plasticity

In spike-timing-dependent plasticity (STDP), the order and precise temporal interval between presynaptic and postsynaptic spikes determine the sign and magnitude of long-term potentiation (LTP) or depression (LTD). STDP is widely utilized in models of circuit-level plasticity, development, and learning. However, spike timing is just one of several factors (including firing rate, synaptic cooperativity, and depolarization) that govern plasticity induction, and its relative importance varies across synapses and activity regimes. This review summarizes this broader view of plasticity, including the forms and cellular mechanisms for the spike-timing dependence of plasticity, and, the evidence that spike timing is an important determinant of plasticity in?vivo.     

Spatiotemporal specificity of synaptic plasticity: cellular rules and mechanisms

Recent experimental results on spike-timing-dependent plasticity (STDP) and heterosynaptic interaction in various systems have revealed new temporal and spatial properties of activity-dependent synaptic plasticity. These results challenge the conventional understanding of Hebb's rule and raise intriguing questions regarding the fundamental processes of cellular signaling. In this article, I review these new findings that lead to formulation of a new set of cellular rules. Emphasis is on evaluating potential molecular and cellular mechanisms that may underlie the spike-timing window of STDP and different patterns of heterosynaptic modifications. I also highlight several unresolved issues, and suggest future lines of research.     

Messages in spike timing-dependent plasticity: pros and cons

Spike-timing dependent plasticity (STDP), a synaptic modification depending on a relative timing of presynaptic and postsynaptic spikes, has fascinated researchers in the fields of neurophysiology and computational neuroscience, because it is not only conceptually simple or biologically reasonable but is also versatile in neural network simulations. The STDP rule may be valid only under specific conditions, however. We propose herein a method that could find more natural and potent rules of synaptic plasticity.     

Bursts shape the NMDA-R mediated spike timing dependent plasticity curve: role of burst interspike interval and GABAergic inhibition

Spike timing dependent plasticity (STDP) is a synaptic learning rule where the relative timing between the presynaptic and postsynaptic action potentials determines the sign and strength of synaptic plasticity. In its basic form STDP has an asymmetric form which incorporates both persistent increases and persistent decreases in synaptic strength. The basic form of STDP, however, is not a fixed property and depends on the dendritic location. An asymmetric curve is observed in the distal dendrites, whereas a symmetrical one is observed in the proximal ones. A recent computational study has shown that the transition from the asymmetry to symmetry is due to inhibition under certain conditions. Synapses have also been observed to be unreliable at generating plasticity when excitatory postsynaptic potentials and single spikes are paired at low frequencies. Bursts of spikes, however, are reliably signaled because transmitter release is facilitated. This article presents a two-compartment model of the CA1 pyramidal cell. The model is neurophysiologically plausible with its dynamics resulting from the interplay of many ionic and synaptic currents. Plasticity is measured by a deterministic Ca²⁺ dynamics model which measures the instantaneous calcium level and its time course in the dendrite and change the strength of the synapse accordingly. The model is validated to match the asymmetrical form of STDP from the pairing of a presynaptic (dendritic) and postsynaptic (somatic) spikes as observed experimentally. With the parameter set unchanged the model investigates how pairing of bursts with single spikes and bursts in the presence or absence of inhibition shapes the STDP curve. The model predicts that inhibition strength and frequency are not the only factors of the asymmetry-to-symmetry switch of the STDP curve. Burst interspike interval is another factor. This study is an important first step towards understanding how STDP is affected under natural firing patterns in vivo.     

The Effect of STDP Temporal Kernel Structure on the Learning Dynamics of Single Excitatory and Inhibitory Synapses

Spike-Timing Dependent Plasticity (STDP) is characterized by a wide range of temporal kernels. However, much of the theoretical work has focused on a specific kernel – the “temporally asymmetric Hebbian” learning rules. Previous studies linked excitatory STDP to positive feedback that can account for the emergence of response selectivity. Inhibitory plasticity was associated with negative feedback that can balance the excitatory and inhibitory inputs. Here we study the possible computational role of the temporal structure of the STDP. We represent the STDP as a superposition of two processes: potentiation and depression. This allows us to model a wide range of experimentally observed STDP kernels, from Hebbian to anti-Hebbian, by varying a single parameter. We investigate STDP dynamics of a single excitatory or inhibitory synapse in purely feed-forward architecture. We derive a mean-field-Fokker-Planck dynamics for the synaptic weight and analyze the effect of STDP structure on the fixed points of the mean field dynamics. We find a phase transition along the Hebbian to anti-Hebbian parameter from a phase that is characterized by a unimodal distribution of the synaptic weight, in which the STDP dynamics is governed by negative feedback, to a phase with positive feedback characterized by a bimodal distribution. The critical point of this transition depends on general properties of the STDP dynamics and not on the fine details. Namely, the dynamics is affected by the pre-post correlations only via a single number that quantifies its overlap with the STDP kernel. We find that by manipulating the STDP temporal kernel, negative feedback can be induced in excitatory synapses and positive feedback in inhibitory. Moreover, there is an exact symmetry between inhibitory and excitatory plasticity, i.e., for every STDP rule of inhibitory synapse there exists an STDP rule for excitatory synapse, such that their dynamics is identical.     

Coactivation of pre- and postsynaptic signaling mechanisms determines cell-specific spike-timing-dependent plasticity

Synapses may undergo long-term increases or decreases in synaptic strength dependent on critical differences in the timing between pre-and postsynaptic activity. Such spike-timing-dependent plasticity (STDP) follows rules that govern how patterns of neural activity induce changes in synaptic strength. Synaptic plasticity in the dorsal cochlear nucleus (DCN) follows Hebbian and anti-Hebbian patterns in a cell-specific manner. Here we show that these opposing responses to synaptic activity result from differential expression of two signaling pathways. Ca2+/calmodulin-dependent protein kinase II (CaMKII) signaling underlies Hebbian postsynaptic LTP in principal cells. By contrast, in interneurons, a temporally precise anti-Hebbian synaptic spike-timing rule results from the combined effects of postsynaptic CaMKII-dependent LTP and endocannabinoid-dependent presynaptic LTD. Cell specificity in the circuit arises from selective targeting of presynaptic CB1 receptors in different axonal terminals. Hence, pre- and postsynaptic sites of expression determine both the sign and timing requirements of long-term plasticity in interneurons.     

Maturation of GABAergic Inhibition Promotes Strengthening of Temporally Coherent Inputs among Convergent Pathways

Spike-timing-dependent plasticity (STDP), a form of Hebbian plasticity, is inherently stabilizing. Whether and how GABAergic inhibition influences STDP is not well understood. Using a model neuron driven by converging inputs modifiable by STDP, we determined that a sufficient level of inhibition was critical to ensure that temporal coherence (correlation among presynaptic spike times) of synaptic inputs, rather than initial strength or number of inputs within a pathway, controlled postsynaptic spike timing. Inhibition exerted this effect by preferentially reducing synaptic efficacy, the ability of inputs to evoke postsynaptic action potentials, of the less coherent inputs. In visual cortical slices, inhibition potently reduced synaptic efficacy at ages during but not before the critical period of ocular dominance (OD) plasticity. Whole-cell recordings revealed that the amplitude of unitary IPSCs from parvalbumin positive (Pv+) interneurons to pyramidal neurons increased during the critical period, while the synaptic decay time-constant decreased. In addition, intrinsic properties of Pv+ interneurons matured, resulting in an increase in instantaneous firing rate. Our results suggest that maturation of inhibition in visual cortex ensures that the temporally coherent inputs (e.g. those from the open eye during monocular deprivation) control postsynaptic spike times of binocular neurons, a prerequisite for Hebbian mechanisms to induce OD plasticity.     

Spectral analysis of input spike trains by spike-timing-dependent plasticity

Spike-timing-dependent plasticity (STDP) has been observed in many brain areas such as sensory cortices, where it is hypothesized to structure synaptic connections between neurons. Previous studies have demonstrated how STDP can capture spiking information at short timescales using specific input configurations, such as coincident spiking, spike patterns and oscillatory spike trains. However, the corresponding computation in the case of arbitrary input signals is still unclear. This paper provides an overarching picture of the algorithm inherent to STDP, tying together many previous results for commonly used models of pairwise STDP. For a single neuron with plastic excitatory synapses, we show how STDP performs a spectral analysis on the temporal cross-correlograms between its afferent spike trains. The postsynaptic responses and STDP learning window determine kernel functions that specify how the neuron "sees" the input correlations. We thus denote this unsupervised learning scheme as 'kernel spectral component analysis' (kSCA). In particular, the whole input correlation structure must be considered since all plastic synapses compete with each other. We find that kSCA is enhanced when weight-dependent STDP induces gradual synaptic competition. For a spiking neuron with a "linear" response and pairwise STDP alone, we find that kSCA resembles principal component analysis (PCA). However, plain STDP does not isolate correlation sources in general, e.g., when they are mixed among the input spike trains. In other words, it does not perform independent component analysis (ICA). Tuning the neuron to a single correlation source can be achieved when STDP is paired with a homeostatic mechanism that reinforces the competition between synaptic inputs. Our results suggest that neuronal networks equipped with STDP can process signals encoded in the transient spiking activity at the timescales of tens of milliseconds for usual STDP.     

A Biophysical Basis for the Inter-spike Interaction of Spike-timing-dependent Plasticity

Although spike-timing-dependent plasticity (STDP) is well characterized when pre- and postsynaptic spikes are paired with a given time lag, how this generalizes for more complex spike-trains is unclear. Recent experiments demonstrate that contributions to synaptic plasticity from different spike pairs within a spike train do not add linearly. In the visual cortex conditioning with spike triplets shows that the effect of the first spike pair dominates over the second. Using a previously proposed calcium-dependent plasticity model, we show that short-term synaptic dynamics and interaction between successive back-propagating action potentials (BPAP) may jointly account for the nonlinearities observed. Paired-pulse depression and attenuation of BPAPs are incorporated into the model through the use-dependent depletion of pre- and postsynaptic resources, respectively. Simulations suggest that these processes may play critical roles in determining how STDP operates in the context of natural spike-trains.     

The Convallis Rule for Unsupervised Learning in Cortical Networks

The phenomenology and cellular mechanisms of cortical synaptic plasticity are becoming known in increasing detail, but the computational principles by which cortical plasticity enables the development of sensory representations are unclear. Here we describe a framework for cortical synaptic plasticity termed the “Convallis rule”, mathematically derived from a principle of unsupervised learning via constrained optimization. Implementation of the rule caused a recurrent cortex-like network of simulated spiking neurons to develop rate representations of real-world speech stimuli, enabling classification by a downstream linear decoder. Applied to spike patterns used in in vitro plasticity experiments, the rule reproduced multiple results including and beyond STDP. However STDP alone produced poorer learning performance. The mathematical form of the rule is consistent with a dual coincidence detector mechanism that has been suggested by experiments in several synaptic classes of juvenile neocortex. Based on this confluence of normative, phenomenological, and mechanistic evidence, we suggest that the rule may approximate a fundamental computational principle of the neocortex.