Spike propagation synchronized by temporally asymmetric Hebbian learning

 Synchronously spiking neurons have been observed in the cerebral cortex and the hippocampus. In computer models, synchronous spike volleys may be propagated across appropriately connected neuron populations. However, it is unclear how the appropriate synaptic connectivity is set up during development and maintained during adult learning. We performed computer simulations to investigate the influence of temporally asymmetric Hebbian synaptic plasticity on the propagation of spike volleys. In addition to feedforward connections, recurrent connections were included between and within neuron populations and spike transmission delays varied due to axonal, synaptic and dendritic transmission. We found that repeated presentations of input volleys decreased the synaptic conductances of intragroup and feedback connections while synaptic conductances of feedforward connections with short delays became stronger than those of connections with longer delays. These adaptations led to the synchronization of spike volleys as they propagated across neuron populations. The findings suggests that temporally asymmetric Hebbian learning may enhance synchronized spiking within small populations of neurons in cortical and hippocampal areas and familiar stimuli may produce synchronized spike volleys that are rapidly propagated across neural tissue. Received: 28 May 2002 / Accepted: 3 June 2002 RID="*" ID="*" Correspondence to: R. E. Suri Intelligent Optical Systems (IOS), 2520 W 237th St, Torrance, CA 90505-5217, USA (e-mail: rsuri@intopsys.com, Tel.: +1-310-5307130 ext. 108, Fax: +1-210-5307417)     

Spatio-temporal credit assignment in neuronal population learning

In learning from trial and error, animals need to relate behavioral decisions to environmental reinforcement even though it may be difficult to assign credit to a particular decision when outcomes are uncertain or subject to delays. When considering the biophysical basis of learning, the credit-assignment problem is compounded because the behavioral decisions themselves result from the spatio-temporal aggregation of many synaptic releases. We present a model of plasticity induction for reinforcement learning in a population of leaky integrate and fire neurons which is based on a cascade of synaptic memory traces. Each synaptic cascade correlates presynaptic input first with postsynaptic events, next with the behavioral decisions and finally with external reinforcement. For operant conditioning, learning succeeds even when reinforcement is delivered with a delay so large that temporal contiguity between decision and pertinent reward is lost due to intervening decisions which are themselves subject to delayed reinforcement. This shows that the model provides a viable mechanism for temporal credit assignment. Further, learning speeds up with increasing population size, so the plasticity cascade simultaneously addresses the spatial problem of assigning credit to synapses in different population neurons. Simulations on other tasks, such as sequential decision making, serve to contrast the performance of the proposed scheme to that of temporal difference-based learning. We argue that, due to their comparative robustness, synaptic plasticity cascades are attractive basic models of reinforcement learning in the brain.     

Further characterization of an aversive learning task in <Emphasis Type="Italic">Drosophila melanogaster</Emphasis>: intensity of the stimulus,relearning, and use of <Emphasis Type="Italic">rutabaga</Emphasis> mutants

Various learning tasks have been described in Drosophila melanogaster, flies being either tested in groups or at the individual level. Le Bourg and Buecher (Anim Learn Behav 33:330–341, 2002) have designed a task at the individual level: photopositive flies crossing a T-maze learn to prefer the dark exit when the lighted one is associated with the presence of aversive stimuli (humidity and quinine). Previous studies have reported various results (e.g. no effect of age) and the present article further characterizes this task by studying the possible effects of: (1) the intensity of the stimuli (quantity of water or concentration of quinine), (2) various delays between two learning sessions on the learning score at the second session, (3) the rutabaga learning mutation on the learning score. More concentrated quinine solutions increased learning scores but the quantity of water had no effect. Learning scores at the second session were higher with shorter delays between the two learning sessions and retrograde amnesia could decrease this memory score. rutabaga mutants showed learning deficits as in experiments testing groups of flies. This learning task could particularly be used to verify whether learning mutants isolated after experiments testing flies in groups display similar deficits when tested at the individual level.     

On the role of anatomy in learning by the cerebellar cortex

The properties due to the location of neurons, synapses, and possibly even synaptic channels, in neuron networks are still unknown. Our preliminary results suggest that not only the interconnections but also the relative positions of the different elements in the network are of importance in the learning process in the cerebellar cortex. We have used neural field equations to investigate the mechanisms of learning in the hierarchical neural network. The numerical resolution of these equations reveals two important properties: (i) The hierarchical structure of this network has the expected effect on learning because the flow of information at the neuronal level is controlled by the heterosynaptic effect through the synaptic density-connectivity function, i.e. the action potential field variable is controlled by the synaptic efficacy field variable at different points of the neuron. (ii) The geometry of the system involves different velocities of propagation along different fibers, i.e. different delays between cells, and thus has a stabilizing effect on the dynamics, allowing the Purkinje output to reach a given value. The field model proposed should be useful in the study of the spatial properties of hierarchical biological systems.     

Associative neural network model for the generation of temporal patterns. Theory and application to central pattern generators.

Cyclic patterns of motor neuron activity are involved in the production of many rhythmic movements, such as walking, swimming, and scratching. These movements are controlled by neural circuits referred to as central pattern generators (CPGs). Some of these circuits function in the absence of both internal pacemakers and external feedback. We describe an associative neural network model whose dynamic behavior is similar to that of CPGs. The theory predicts the strength of all possible connections between pairs of neurons on the basis of the outputs of the CPG. It also allows the mean operating levels of the neurons to be deduced from the measured synaptic strengths between the pairs of neurons. We apply our theory to the CPG controlling escape swimming in the mollusk Tritonia diomedea. The basic rhythmic behavior is shown to be consistent with a simplified model that approximates neurons as threshold units and slow synaptic responses as elementary time delays. The model we describe may have relevance to other fixed action behaviors, as well as to the learning, recall, and recognition of temporally ordered information.     

A neural network model of the cerebellar cortex performing dynamic associations

The present paper proposes a model which applies formal neural network modeling techniques to construct a theoretical representation of the cerebellar cortex and its performances in motor control. A schema that makes explicit use of propagation delays of neural signals, is introduced to describe the ability to store temporal sequences of patterns in the Golgi-granule cell system. A perceptron association is then performed on these sequences of patterns by the Purkinje cell layer. The model conforms with important biological constraints, such as the known excitatory or inhibitory nature of the various synapses. Also, as suggested by experimental evidence, the synaptic plasticity underlying the learning ability of the model, is confined to the parallel fiber — Purkinje cell synapses, and takes place under the control of the climbing fibers. The result is a neural network model, constructed according to the anatomy of the cerebellar cortex, and capable of learning and retrieval of temporal sequences of patterns. It provides a framework to represent and interpret properties of learning and control of movements by the cerebellum, and to assess the capacity of formal neural network techniques for modeling of real neural systems.     

A Biophysical Model of Synaptic Delay Learning and Temporal Pattern Recognition in a Cerebellar Purkinje Cell

It has been suggested that information in the brain is encoded in temporal spike patterns which are decoded by a combination of time delays and coincidence detection. Here, we show how a multi-compartmental model of a cerebellar Purkinje cell can learn to recognise temporal parallel fibre activity patterns by adapting latencies of calcium responses after activation of metabotropic glutamate receptors (mGluRs). In each compartment of our model, the mGluR signalling cascade is represented by a set of differential equations that reflect the underlying biochemistry. Phosphorylation of the mGluRs changes the concentration of receptors which are available for activation by glutamate and thereby adjusts the time delay between mGluR stimulation and voltage response. The adaptation of a synaptic delay as opposed to a weight represents a novel non-Hebbian learning mechanism that can also implement the adaptive timing of the classically conditioned eye-blink response.     

Computer simulation of inhibition-dependent binding in a neural network

Reverberating dynamics of neural network is modeled on PC in order to illustrate possible role of inhibition as binding controller in the network. The network is composed of binding neurons. In the binding neuron model [BioSystems 48 (1998) 263], the degree of temporal coherence between synaptic inputs is decisive for triggering, and slow inhibition is expressed in terms of the degree, which is necessary for triggering. Two learning mechanisms are implemented in the network, namely, adjusting synaptic strength and/or propagation delays. By means of forced playing of external pattern, the network is taught to support dynamics with disconnected and bound patterns of activity. By choosing either high, or low inhibition, one can switch between the disconnected and bound patterns, respectively. This is interpreted as inhibition-controlled binding in the network.     

How Attention Can Create Synaptic Tags for the Learning of Working Memories in Sequential Tasks

Intelligence is our ability to learn appropriate responses to new stimuli and situations. Neurons in association cortex are thought to be essential for this ability. During learning these neurons become tuned to relevant features and start to represent them with persistent activity during memory delays. This learning process is not well understood. Here we develop a biologically plausible learning scheme that explains how trial-and-error learning induces neuronal selectivity and working memory representations for task-relevant information. We propose that the response selection stage sends attentional feedback signals to earlier processing levels, forming synaptic tags at those connections responsible for the stimulus-response mapping. Globally released neuromodulators then interact with tagged synapses to determine their plasticity. The resulting learning rule endows neural networks with the capacity to create new working memory representations of task relevant information as persistent activity. It is remarkably generic: it explains how association neurons learn to store task-relevant information for linear as well as non-linear stimulus-response mappings, how they become tuned to category boundaries or analog variables, depending on the task demands, and how they learn to integrate probabilistic evidence for perceptual decisions.     

Tuning bat LSO neurons to interaural intensity differences through spike-timing dependent plasticity

Bats, like other mammals, are known to use interaural intensity differences (IID) to determine azimuthal position. In the lateral superior olive (LSO) neurons have firing behaviors which vary systematically with IID. Those neurons receive excitatory inputs from the ipsilateral ear and inhibitory inputs from the contralateral one. The IID sensitivity of a LSO neuron is thought to be due to delay differences between the signals coming from both ears, differences due to different synaptic delays and to intensity-dependent delays. In this paper we model the auditory pathway until the LSO. We propose a learning scheme where inputs to LSO neurons start out numerous with different relative delays. Spike timing-dependent plasticity (STDP) is then used to prune those connections. We compare the pruned neuron responses with physiological data and analyse the relationship between IID’s of teacher stimuli and IID sensitivities of trained LSO neurons.     

Emergence of network structure due to spike-timing-dependent plasticity in recurrent neuronal networks V: self-organization schemes and weight dependence

Spike-timing-dependent plasticity (STDP) determines the evolution of the synaptic weights according to their pre- and post-synaptic activity, which in turn changes the neuronal activity on a (much) slower time scale. This paper examines the effect of STDP in a recurrently connected network stimulated by external pools of input spike trains, where both input and recurrent synapses are plastic. Our previously developed theoretical framework is extended to incorporate weight-dependent STDP and dendritic delays. The weight dynamics is determined by an interplay between the neuronal activation mechanisms, the input spike-time correlations, and the learning parameters. For the case of two external input pools, the resulting learning scheme can exhibit a symmetry breaking of the input connections such that two neuronal groups emerge, each specialized to one input pool only. In addition, we show how the recurrent connections within each neuronal group can be strengthened by STDP at the expense of those between the two groups. This neuronal self-organization can be seen as a basic dynamical ingredient for the emergence of neuronal maps induced by activity-dependent plasticity.     

An STDP training algorithm for a spiking neural network with dynamic threshold neurons

This paper proposes a supervised training algorithm for Spiking Neural Networks (SNNs) which modifies the Spike Timing Dependent Plasticity (STDP)learning rule to support both local and network level training with multiple synaptic connections and axonal delays. The training algorithm applies the rule to two and three layer SNNs, and is benchmarked using the Iris and Wisconsin Breast Cancer (WBC) data sets. The effectiveness of hidden layer dynamic threshold neurons is also investigated and results are presented.     

Stability of complex spike timing-dependent plasticity in cerebellar learning

Dynamics of spike-timing dependent synaptic plasticity are analyzed for excitatory and inhibitory synapses onto cerebellar Purkinje cells. The purpose of this study is to place theoretical constraints on candidate synaptic learning rules that determine the changes in synaptic efficacy due to pairing complex spikes with presynaptic spikes in parallel fibers and inhibitory interneurons. Constraints are derived for the timing between complex spikes and presynaptic spikes, constraints that result from the stability of the learning dynamics of the learning rule. Potential instabilities in the parallel fiber synaptic learning rule are found to be stabilized by synaptic plasticity at inhibitory synapses if the inhibitory learning rules are stable, and conditions for stability of inhibitory plasticity are given. Combining excitatory with inhibitory plasticity provides a mechanism for minimizing the overall synaptic input. Stable learning rules are shown to be able to sculpt simple-spike patterns by regulating the excitability of neurons in the inferior olive that give rise to climbing fibers.     

Beyond spike timing: the role of nonlinear plasticity and unreliable synapses

 Spike-timing-dependent plasticity (STDP) strengthens synapses that are activated immediately before a postsynaptic spike, and weakens those that are activated after a spike. To prevent an uncontrolled growth of the synaptic strengths, weakening must dominate strengthening for uncorrelated spike times. However, this weight-normalization property would preclude Hebbian potentiation when the pre- and postsynaptic neurons are strongly active without specific spike-time correlations. We show that nonlinear STDP as inherent in the data of Markram et al. [(1997) Science 275:213–215] can preserve the benefits of both weight normalization and Hebbian plasticity, and hence can account for learning based on spike-time correlations and on mean firing rates. As examples we consider the moving-threshold property of the Bienenstock–Cooper–Munro rule, the development of direction-selective simple cells by changing short-term synaptic depression, and the joint adaptation of axonal and dendritic delays. Without threshold nonlinearity at low frequencies, the development of direction selectivity does not stabilize in a natural stimulation environment. Without synaptic unreliability there is no causal development of axonal and dendritic delays. Received: 22 April 2002 / Accepted: 23 May 2002 Acknowledgements. This study was supported by the Swiss National Science Foundation (grant 3152-065234.01) and the Silva-Casa foundation. The author thanks Stefano Fusi, Henry Markram, and Misha Tsodyks for helpful discussions, Nissim Buchs and Martin Schneider for their simulations, and Jan Reutimann for proof reading. Correspondence to: e-mail: wsenn@cns.unibe.ch, Tel.: +41-31-6318721, Fax: 41-31-6314611     

Presynaptic plasticity and associative learning are impaired in a Drosophila presenilin null mutant

Alzheimer's disease is a neurodegenerative disorder characterized by progressive memory and cognitive decline that is associated with changes in synaptic plasticity and neuronal cell loss. Recent evidence suggests that some of these defects may be due to a loss of normal presenilin activity. Here, we have examined the effect of loss of Drosophila presenilin (psn) function on synaptic plasticity and learning. Basal transmitter release was elevated in psn mutants while both paired pulse synaptic plasticity and post-tetanic potentiation were impaired. These defects in synaptic strength and plasticity were not due to developmental defects in NMJ morphology. We also found that psn null terminals take up significantly less FM 4-64 than control terminals when loaded with high frequency stimulation, suggesting a defect in synaptic vesicle availability or mobilization. To determine whether these reductions in synaptic plasticity had any impact on learning, we tested the larvae for defects in associative learning. Using both olfactory and visual learning assays, we found that associative learning is impaired in psn mutants compared with controls. Both the learning and synaptic defects could be rescued by expression of a full length psn transgene suggesting the defects are specifically due to a loss of psn function. Taken together, these results provide the first evidence of learning and synaptic defects in a Drosophila psn mutant and strongly suggest a presynaptic role for presenilin in normal neuronal function.     

Falling towards forgetfulness: synaptic decay prevents spontaneous recovery of memory

Long after a new language has been learned and forgotten, relearning a few words seems to trigger the recall of other words. This "free-lunch learning" (FLL) effect has been demonstrated both in humans and in neural network models. Specifically, previous work proved that linear networks that learn a set of associations, then partially forget them all, and finally relearn some of the associations, show improved performance on the remaining (i.e., nonrelearned) associations. Here, we prove that relearning forgotten associations decreases performance on nonrelearned associations; an effect we call negative free-lunch learning. The difference between free-lunch learning and the negative free-lunch learning presented here is due to the particular method used to induce forgetting. Specifically, if forgetting is induced by isotropic drifting of weight vectors (i.e., by adding isotropic noise), then free-lunch learning is observed. However, as proved here, if forgetting is induced by weight values that simply decay or fall towards zero, then negative free-lunch learning is observed. From a biological perspective, and assuming that nervous systems are analogous to the networks used here, this suggests that evolution may have selected physiological mechanisms that involve forgetting using a form of synaptic drift rather than synaptic decay, because synaptic drift, but not synaptic decay, yields free-lunch learning.     

Need probability effects in animal short-term memory

Five pigeons performed in a delayed matching-to-sample (DMTS) procedure with five delay durations (0.5, 2.5, 5, 10 and 20 s) mixed within sessions. Contrary to the predictions of need probability theory, discriminability decreased when fewer short than long delays were included in each session. To test whether the decrease in discriminability was due to a decrease in obtained reinforcement at short delays, the number of trials at each delay was held constant and reinforcer probability was increased with increasing delay. This manipulation produced a similar decrease in discriminability as when the frequency of delays was manipulated. It was concluded that the effect of delay frequency on the forgetting function is mediated by the effect of the reinforcer distribution, which influences discriminability by weakening stimulus control.     

Interactions between NEEP21, GRIP1 and GluR2 regulate sorting and recycling of the glutamate receptor subunit GluR2

Trafficking of AMPA-type glutamate receptors (AMPAR) between endosomes and the postsynaptic plasma membrane of neurons plays a central role in the control of synaptic strength associated with learning and memory. The molecular mechanisms of its regulation remain poorly understood, however. Here we show by biochemical and atomic force microscopy analyses that NEEP21, a neuronal endosomal protein necessary for receptor recycling including AMPAR, is associated with the scaffolding protein GRIP1 and the AMPAR subunit GluR2. Moreover, the interaction between NEEP21 and GRIP1 is regulated by neuronal activity. Expression of a NEEP21 fragment containing the GRIP1-binding site decreases surface GluR2 levels and delays recycling of internalized GluR2, which accumulates in early endosomes and lysosomes. Infusion of this fragment into pyramidal neurons of hippocampal slices induces inward rectification of AMPAR-mediated synaptic responses, suggesting decreased GluR2 expression at synapses. These results indicate that NEEP21-GRIP1 binding is crucial for GluR2-AMPAR sorting through endosomes and their recruitment to the plasma membrane, providing a first molecular mechanism to differentially regulate AMPAR subunit cycling in internal compartments.     

Somato-dendritic Synaptic Plasticity and Error-backpropagation in Active Dendrites

In the last decade dendrites of cortical neurons have been shown to nonlinearly combine synaptic inputs by evoking local dendritic spikes. It has been suggested that these nonlinearities raise the computational power of a single neuron, making it comparable to a 2-layer network of point neurons. But how these nonlinearities can be incorporated into the synaptic plasticity to optimally support learning remains unclear. We present a theoretically derived synaptic plasticity rule for supervised and reinforcement learning that depends on the timing of the presynaptic, the dendritic and the postsynaptic spikes. For supervised learning, the rule can be seen as a biological version of the classical error-backpropagation algorithm applied to the dendritic case. When modulated by a delayed reward signal, the same plasticity is shown to maximize the expected reward in reinforcement learning for various coding scenarios. Our framework makes specific experimental predictions and highlights the unique advantage of active dendrites for implementing powerful synaptic plasticity rules that have access to downstream information via backpropagation of action potentials.     

Learning the same motor task twice impairs its retention in a time- and dose-dependent manner

Anterograde interference emerges when two differing tasks are learned in close temporal proximity, an effect repeatedly attributed to a competition between differing task memories. However, recent development alternatively suggests that initial learning may trigger a refractory period that occludes neuroplasticity and impairs subsequent learning, consequently mediating interference independently of memory competition. Accordingly, this study tested the hypothesis that interference can emerge when the same motor task is being learned twice, that is when competition between memories is prevented. In a first experiment, the inter-session interval (ISI) between two identical motor learning sessions was manipulated to be 2 min, 1 h or 24 h. Results revealed that retention of the second session was impaired as compared to the first one when the ISI was 2 min but not when it was 1 h or 24 h, indicating a time-dependent process. Results from a second experiment replicated those of the first one and revealed that adding a third motor learning session with a 2 min ISI further impaired retention, indicating a dose-dependent process. Results from a third experiment revealed that the retention impairments did not take place when a learning session was preceded by simple rehearsal of the motor task without concurrent learning, thus ruling out fatigue and confirming that retention is impaired specifically when preceded by a learning session. Altogether, the present results suggest that competing memories is not the sole mechanism mediating anterograde interference and introduce the possibility that a time- and dose-dependent refractory period—independent of fatigue—also contributes to its emergence. One possibility is that learning transiently perturbs the homeostasis of learning-related neuronal substrates. Introducing additional learning when homeostasis is still perturbed may not only impair performance improvements, but also memory formation.