Hebbian learning in parallel and modular memories

Many cognitive and sensorimotor functions in the brain involve parallel and modular memory subsystems that are adapted by activity-dependent Hebbian synaptic plasticity. This is in contrast to the multilayer perceptron model of supervised learning where sensory information is presumed to be integrated by a common pool of hidden units through backpropagation learning. Here we show that Hebbian learning in parallel and modular memories is more advantageous than backpropagation learning in lumped memories in two respects: it is computationally much more efficient and structurally much simpler to implement with biological neurons. Accordingly, we propose a more biologically relevant neural network model, called a tree-like perceptron, which is a simple modification of the multilayer perceptron model to account for the general neural architecture, neuronal specificity, and synaptic learning rule in the brain. The model features a parallel and modular architecture in which adaptation of the input-to-hidden connection follows either a Hebbian or anti-Hebbian rule depending on whether the hidden units are excitatory or inhibitory, respectively. The proposed parallel and modular architecture and implicit interplay between the types of synaptic plasticity and neuronal specificity are exhibited by some neocortical and cerebellar systems. Received: 13 October 1996 / Accepted in revised form: 16 October 1997     

Emergence of symmetric, modular, and reciprocal connections in recurrent networks with Hebbian learning

While learning and development are well characterized in feedforward networks, these features are more difficult to analyze in recurrent networks due to the increased complexity of dual dynamics – the rapid dynamics arising from activation states and the slow dynamics arising from learning or developmental plasticity. We present analytical and numerical results that consider dual dynamics in a recurrent network undergoing Hebbian learning with either constant weight decay or weight normalization. Starting from initially random connections, the recurrent network develops symmetric or near-symmetric connections through Hebbian learning. Reciprocity and modularity arise naturally through correlations in the activation states. Additionally, weight normalization may be better than constant weight decay for the development of multiple attractor states that allow a diverse representation of the inputs. These results suggest a natural mechanism by which synaptic plasticity in recurrent networks such as cortical and brainstem premotor circuits could enhance neural computation and the generation of motor programs. Received: 27 April 1998 / Accepted in revised form: 16 March 1999     

Development of feature detectors by self-organization

We present a two-layered network of linear neurons that organizes itself as to extract the complete information contained in a set of presented patterns. The weights between layers obey a Hebbian rule. We propose a local anti-Hebbian rule for lateral, hierarchically organized weights within the output layer. This rule forces the activities of the output units to become uncorrelated and the lateral weights to vanish. The weights between layers converge to the eigenvectors of the covariance matrix of input patterns, i.e., the network performs a principal component analysis, yielding all principal components. As a consequence of the proposed learning scheme, the output units become detectors of orthogonal features, similar to ones found in the brain of mammals.     

A Developmental Switch for Hebbian Plasticity

Hebbian forms of synaptic plasticity are required for the orderly development of sensory circuits in the brain and are powerful modulators of learning and memory in adulthood. During development, emergence of Hebbian plasticity leads to formation of functional circuits. By modeling the dynamics of neurotransmitter release during early postnatal cortical development we show that a developmentally regulated switch in vesicle exocytosis mode triggers associative (i.e. Hebbian) plasticity. Early in development spontaneous vesicle exocytosis (SVE), often considered as ''synaptic noise'', is important for homogenization of synaptic weights and maintenance of synaptic weights in the appropriate dynamic range. Our results demonstrate that SVE has a permissive, whereas subsequent evoked vesicle exocytosis (EVE) has an instructive role in the expression of Hebbian plasticity. A timed onset for Hebbian plasticity can be achieved by switching from SVE to EVE and the balance between SVE and EVE can control the effective rate of Hebbian plasticity. We further show that this developmental switch in neurotransmitter release mode enables maturation of spike-timing dependent plasticity. A mis-timed or inadequate SVE to EVE switch may lead to malformation of brain networks thereby contributing to the etiology of neurodevelopmental disorders.     

Beyond spike timing: the role of nonlinear plasticity and unreliable synapses

 Spike-timing-dependent plasticity (STDP) strengthens synapses that are activated immediately before a postsynaptic spike, and weakens those that are activated after a spike. To prevent an uncontrolled growth of the synaptic strengths, weakening must dominate strengthening for uncorrelated spike times. However, this weight-normalization property would preclude Hebbian potentiation when the pre- and postsynaptic neurons are strongly active without specific spike-time correlations. We show that nonlinear STDP as inherent in the data of Markram et al. [(1997) Science 275:213–215] can preserve the benefits of both weight normalization and Hebbian plasticity, and hence can account for learning based on spike-time correlations and on mean firing rates. As examples we consider the moving-threshold property of the Bienenstock–Cooper–Munro rule, the development of direction-selective simple cells by changing short-term synaptic depression, and the joint adaptation of axonal and dendritic delays. Without threshold nonlinearity at low frequencies, the development of direction selectivity does not stabilize in a natural stimulation environment. Without synaptic unreliability there is no causal development of axonal and dendritic delays. Received: 22 April 2002 / Accepted: 23 May 2002 Acknowledgements. This study was supported by the Swiss National Science Foundation (grant 3152-065234.01) and the Silva-Casa foundation. The author thanks Stefano Fusi, Henry Markram, and Misha Tsodyks for helpful discussions, Nissim Buchs and Martin Schneider for their simulations, and Jan Reutimann for proof reading. Correspondence to: e-mail: wsenn@cns.unibe.ch, Tel.: +41-31-6318721, Fax: 41-31-6314611     

Kinetic models of spike-timing dependent plasticity and their functional consequences in detecting correlations

Spike-timing dependent plasticity (STDP) is a type of synaptic modification found relatively recently, but the underlying biophysical mechanisms are still unclear. Several models of STDP have been proposed, and differ by their implementation, and in particular how synaptic weights saturate to their minimal and maximal values. We analyze here kinetic models of transmitter-receptor interaction and derive a series of STDP models. In general, such kinetic models predict progressive saturation of the weights. Various forms can be obtained depending on the hypotheses made in the kinetic model, and these include a simple linear dependence on the value of the weight (“soft bounds”), mixed soft and abrupt saturation (“hard bound”), or more complex forms. We analyze in more detail simple soft-bound models of Hebbian and anti-Hebbian STDPs, in which nonlinear spike interactions (triplets) are taken into account. We show that Hebbian STDPs can be used to selectively potentiate synapses that are correlated in time, while anti-Hebbian STDPs depress correlated synapses, despite the presence of nonlinear spike interactions. This correlation detection enables neurons to develop a selectivity to correlated inputs. We also examine different versions of kinetics-based STDP models and compare their sensitivity to correlations. We conclude that kinetic models generally predict soft-bound dynamics, and that such models seem ideal for detecting correlations among large numbers of inputs.     

Hebbian errors in learning: An analysis using the Oja model

Background: Recent work on long term potentiation in brain slices shows that Hebb's rule is not completely synapse-specific, probably due to intersynapse diffusion of calcium or other factors. We previously suggested that such errors in Hebbian learning might be analogous to mutations in evolution.Methods and findings: We examine this proposal quantitatively, extending the classical Oja unsupervised model of learning by a single linear neuron to include Hebbian inspecificity. We introduce an error matrix E, which expresses possible crosstalk between updating at different connections. When there is no inspecificity, this gives the classical result of convergence to the first principal component of the input distribution (PC1). We show the modified algorithm converges to the leading eigenvector of the matrix EC, where C is the input covariance matrix. In the most biologically plausible case when there are no intrinsically privileged connections, E has diagonal elements Q and off-diagonal elements (1-Q)/(n-1), where Q, the quality, is expected to decrease with the number of inputs n and with a synaptic parameter b that reflects synapse density, calcium diffusion, etc. We study the dependence of the learning accuracy on b, n and the amount of input activity or correlation (analytically and computationally). We find that accuracy increases (learning becomes gradually less useful) with increases in b, particularly for intermediate (i.e., biologically realistic) correlation strength, although some useful learning always occurs up to the trivial limit Q=1/n.Conclusions and significance: We discuss the relation of our results to Hebbian unsupervised learning in the brain. When the mechanism lacks specificity, the network fails to learn the expected, and typically most useful, result, especially when the input correlation is weak. Hebbian crosstalk would reflect the very high density of synapses along dendrites, and inevitably degrades learning.     

Fast heterosynaptic learning in a robot food retrieval task inspired by the limbic system

Hebbian learning is the most prominent paradigm in correlation based learning: if pre- and postsynaptic activity coincides the weight of the synapse is strengthened. Hebbian learning however, is not stable because of an autocorrelation term which causes the weights to grow exponentially. The standard solution would be to compensate the autocorrelation term. However, in this work we present a heterosynaptic learning rule which does not have an autocorrelation term and therefore does not show the instability of Hebbian learning. Consequently our heterosynaptic learning is much more stable than the classical Hebbian learning. The performance of our learning rule is demonstrated in a model which is inspired by the limbic system where an agent has to retrieve food.     

Balancing feed-forward excitation and inhibition via Hebbian inhibitory synaptic plasticity

It has been suggested that excitatory and inhibitory inputs to cortical cells are balanced, and that this balance is important for the highly irregular firing observed in the cortex. There are two hypotheses as to the origin of this balance. One assumes that it results from a stable solution of the recurrent neuronal dynamics. This model can account for a balance of steady state excitation and inhibition without fine tuning of parameters, but not for transient inputs. The second hypothesis suggests that the feed forward excitatory and inhibitory inputs to a postsynaptic cell are already balanced. This latter hypothesis thus does account for the balance of transient inputs. However, it remains unclear what mechanism underlies the fine tuning required for balancing feed forward excitatory and inhibitory inputs. Here we investigated whether inhibitory synaptic plasticity is responsible for the balance of transient feed forward excitation and inhibition. We address this issue in the framework of a model characterizing the stochastic dynamics of temporally anti-symmetric Hebbian spike timing dependent plasticity of feed forward excitatory and inhibitory synaptic inputs to a single post-synaptic cell. Our analysis shows that inhibitory Hebbian plasticity generates 'negative feedback' that balances excitation and inhibition, which contrasts with the 'positive feedback' of excitatory Hebbian synaptic plasticity. As a result, this balance may increase the sensitivity of the learning dynamics to the correlation structure of the excitatory inputs.     

Input-dependent learning rule for the memory of spatiotemporal sequences in hippocampal network with theta phase precession

Theta phase precession in rat hippocampal place cells is hypothesized to contribute to memory encoding of running experience in the sense that it provides the ideal timing for synaptic plasticity and enables the asymmetric associative connections under the Hebbian learning rule with asymmetric time window (Yamaguchi 2003). When the sequence of place fields is considered as the episodic memory of running experience, a given spatial route should be accurately stored in spite of differing overlap extent among place fields and varying running velocity. Using a hippocampal network model with phase precession and the Hebbian learning rule with asymmetric time window, we investigate the memory encoding of place field sequences in a single traversal experience. Computer experiments show that place fields cannot be stored correctly until an input-dependent feature is introduced into the learning rule. These experiments further indicate that there exists an optimum value for the saturation level of synaptic plasticity and the speed of synaptic plasticity in the learning rule, which are correlated with, respectively, the overlap extent of place field sequence and the running velocity of animal during traversal. A comparison of these results with biological evidences shows good agreement and suggests that behavior-dependent regulation of the learning rule is necessary for memory encoding.     

Dual synaptic plasticity in the hippocampus: Hebbian and spatiotemporal learning dynamics

We assume that Hebbian learning dynamics (HLD) and spatiotemporal learning dynamics (SLD) are involved in the mechanism of synaptic plasticity in the hippocampal neurons. While HLD is driven by pre- and postsynaptic spike timings through the backpropagating action potential, SLD is evoked by presynaptic spike timings alone. Since the backpropagation attenuates as it nears the distal dendrites, we assume an extreme case as a neuron model where HLD exists only at proximal dendrites and SLD exists only at the distal dendrites. We examined how the synaptic weights change in response to three types of synaptic inputs in computer simulations. First, in response to a Poisson train having a constant mean frequency, the synaptic weights in HLD and SLD are qualitatively similar. Second, SLD responds more rapidly than HLD to synchronous input patterns, while each responds to them. Third, HLD responds more rapidly to more frequent inputs, while SLD shows fluctuating synaptic weights. These results suggest an encoding hypothesis in that a transient synchronous structure in spatiotemporal input patterns will be encoded into distal dendrites through SLD and that persistent synchrony or firing rate information will be encoded into proximal dendrites through HLD.     

Effects of Hebbian learning on the dynamics and structure of random networks with inhibitory and excitatory neurons

The aim of the present paper is to study the effects of Hebbian learning in random recurrent neural networks with biological connectivity, i.e. sparse connections and separate populations of excitatory and inhibitory neurons. We furthermore consider that the neuron dynamics may occur at a (shorter) time scale than synaptic plasticity and consider the possibility of learning rules with passive forgetting. We show that the application of such Hebbian learning leads to drastic changes in the network dynamics and structure. In particular, the learning rule contracts the norm of the weight matrix and yields a rapid decay of the dynamics complexity and entropy. In other words, the network is rewired by Hebbian learning into a new synaptic structure that emerges with learning on the basis of the correlations that progressively build up between neurons. We also observe that, within this emerging structure, the strongest synapses organize as a small-world network. The second effect of the decay of the weight matrix spectral radius consists in a rapid contraction of the spectral radius of the Jacobian matrix. This drives the system through the "edge of chaos" where sensitivity to the input pattern is maximal. Taken together, this scenario is remarkably predicted by theoretical arguments derived from dynamical systems and graph theory.     

Synaptic Plasticity in Neural Networks Needs Homeostasis with a Fast Rate Detector

Hebbian changes of excitatory synapses are driven by and further enhance correlations between pre- and postsynaptic activities. Hence, Hebbian plasticity forms a positive feedback loop that can lead to instability in simulated neural networks. To keep activity at healthy, low levels, plasticity must therefore incorporate homeostatic control mechanisms. We find in numerical simulations of recurrent networks with a realistic triplet-based spike-timing-dependent plasticity rule (triplet STDP) that homeostasis has to detect rate changes on a timescale of seconds to minutes to keep the activity stable. We confirm this result in a generic mean-field formulation of network activity and homeostatic plasticity. Our results strongly suggest the existence of a homeostatic regulatory mechanism that reacts to firing rate changes on the order of seconds to minutes.     

Natural patterns of activity and long-term synaptic plasticity

Long-term potentiation (LTP) of synaptic transmission is traditionally elicited by massively synchronous, high-frequency inputs, which rarely occur naturally. Recent in vitro experiments have revealed that both LTP and long-term depression (LTD) can arise by appropriately pairing weak synaptic inputs with action potentials in the postsynaptic cell. This discovery has generated new insights into the conditions under which synaptic modification may occur in pyramidal neurons in vivo. First, it has been shown that the temporal order of the synaptic input and the postsynaptic spike within a narrow temporal window determines whether LTP or LTD is elicited, according to a temporally asymmetric Hebbian learning rule. Second, backpropagating action potentials are able to serve as a global signal for synaptic plasticity in a neuron compared with local associative interactions between synaptic inputs on dendrites. Third, a specific temporal pattern of activity--postsynaptic bursting--accompanies synaptic potentiation in adults.     

Hebbian learning and predictive mirror neurons for actions,sensations and emotions

Spike-timing-dependent plasticity is considered the neurophysiological basis of Hebbian learning and has been shown to be sensitive to both contingency and contiguity between pre- and postsynaptic activity. Here, we will examine how applying this Hebbian learning rule to a system of interconnected neurons in the presence of direct or indirect re-afference (e.g. seeing/hearing one''s own actions) predicts the emergence of mirror neurons with predictive properties. In this framework, we analyse how mirror neurons become a dynamic system that performs active inferences about the actions of others and allows joint actions despite sensorimotor delays. We explore how this system performs a projection of the self onto others, with egocentric biases to contribute to mind-reading. Finally, we argue that Hebbian learning predicts mirror-like neurons for sensations and emotions and review evidence for the presence of such vicarious activations outside the motor system.     

The legacy of Donald O. Hebb: more than the Hebb synapse

Neuroscientists associate the name of Donald O. Hebb with the Hebbian synapse and the Hebbian learning rule, which underlie connectionist theories and synaptic plasticity, but Hebb's work has also influenced developmental psychology, neuropsychology, perception and the study of emotions, as well as learning and memory. Here, we review the work of Hebb and its lasting influence on neuroscience in honour of the 2004 centenary of his birth.     

Functional imaging of perceptual learning in human primary and secondary somatosensory cortex

Cellular mechanisms underlying synaptic plasticity are in line with the Hebbian concept. In contrast, data linking Hebbian learning to altered perception are rare. Combining functional magnetic resonance imaging with psychophysical tests, we studied cortical reorganization in primary and secondary somatosensory cortex (SI and SII) and the resulting changes of tactile perception before and after tactile coactivation, a simple type of Hebbian learning. Coactivation on the right index finger (IF) for 3 hr lowered its spatial discrimination threshold. In parallel, blood-oxygen level-dependent (BOLD) signals from the right IF representation in SI and SII enlarged. The individual threshold reduction was linearly correlated with the enlargement in SI, implying a close relation between altered discrimination and cortical reorganization. Controls consisting of a single-site stimulation did not affect thresholds and cortical maps. Accordingly, changes within distributed cortical networks based on Hebbian mechanisms alter the individual percept.     

Self-Organization of Microcircuits in Networks of Spiking Neurons with Plastic Synapses

The synaptic connectivity of cortical networks features an overrepresentation of certain wiring motifs compared to simple random-network models. This structure is shaped, in part, by synaptic plasticity that promotes or suppresses connections between neurons depending on their joint spiking activity. Frequently, theoretical studies focus on how feedforward inputs drive plasticity to create this network structure. We study the complementary scenario of self-organized structure in a recurrent network, with spike timing-dependent plasticity driven by spontaneous dynamics. We develop a self-consistent theory for the evolution of network structure by combining fast spiking covariance with a slow evolution of synaptic weights. Through a finite-size expansion of network dynamics we obtain a low-dimensional set of nonlinear differential equations for the evolution of two-synapse connectivity motifs. With this theory in hand, we explore how the form of the plasticity rule drives the evolution of microcircuits in cortical networks. When potentiation and depression are in approximate balance, synaptic dynamics depend on weighted divergent, convergent, and chain motifs. For additive, Hebbian STDP these motif interactions create instabilities in synaptic dynamics that either promote or suppress the initial network structure. Our work provides a consistent theoretical framework for studying how spiking activity in recurrent networks interacts with synaptic plasticity to determine network structure.     

Self-influencing synaptic plasticity: Recurrent changes of synaptic weights can lead to specific functional properties

Recent experimental results suggest that dendritic and back-propagating spikes can influence synaptic plasticity in different ways (Holthoff, 2004; Holthoff et al., 2005). In this study we investigate how these signals could interact at dendrites in space and time leading to changing plasticity properties at local synapse clusters. Similar to a previous study (Saudargiene et al., 2004) we employ a differential Hebbian learning rule to emulate spike-timing dependent plasticity and investigate how the interaction of dendritic and back-propagating spikes, as the post-synaptic signals, could influence plasticity. Specifically, we will show that local synaptic plasticity driven by spatially confined dendritic spikes can lead to the emergence of synaptic clusters with different properties. If one of these clusters can drive the neuron into spiking, plasticity may change and the now arising global influence of a back-propagating spike can lead to a further segregation of the clusters and possibly the dying-off of some of them leading to more functional specificity. These results suggest that through plasticity being a spatial and temporal local process, the computational properties of dendrites or complete neurons can be substantially augmented. Action Editor: Wulfram Gerstner     

Homeostatic plasticity in the CNS: synaptic and intrinsic forms

The study of experience-dependent plasticity has been dominated by questions of how Hebbian plasticity mechanisms act during learning and development. This is unsurprising as Hebbian plasticity constitutes the most fully developed and influential model of how information is stored in neural circuits and how neural circuitry can develop without extensive genetic instructions. Yet Hebbian plasticity may not be sufficient for understanding either learning or development: the dramatic changes in synapse number and strength that can be produced by this kind of plasticity tend to threaten the stability of neural circuits. Recent work has suggested that, in addition to Hebbian plasticity, homeostatic regulatory mechanisms are active in a variety of preparations. These mechanisms alter both the synaptic connections between neurons and the intrinsic electrical properties of individual neurons, in such a way as to maintain some constancy in neuronal properties despite the changes wrought by Hebbian mechanisms. Here we review the evidence for homeostatic plasticity in the central nervous system, with special emphasis on results from cortical preparations.