The chronotron: a neuron that learns to fire temporally precise spike patterns

In many cases, neurons process information carried by the precise timings of spikes. Here we show how neurons can learn to generate specific temporally precise output spikes in response to input patterns of spikes having precise timings, thus processing and memorizing information that is entirely temporally coded, both as input and as output. We introduce two new supervised learning rules for spiking neurons with temporal coding of information (chronotrons), one that provides high memory capacity (E-learning), and one that has a higher biological plausibility (I-learning). With I-learning, the neuron learns to fire the target spike trains through synaptic changes that are proportional to the synaptic currents at the timings of real and target output spikes. We study these learning rules in computer simulations where we train integrate-and-fire neurons. Both learning rules allow neurons to fire at the desired timings, with sub-millisecond precision. We show how chronotrons can learn to classify their inputs, by firing identical, temporally precise spike trains for different inputs belonging to the same class. When the input is noisy, the classification also leads to noise reduction. We compute lower bounds for the memory capacity of chronotrons and explore the influence of various parameters on chronotrons' performance. The chronotrons can model neurons that encode information in the time of the first spike relative to the onset of salient stimuli or neurons in oscillatory networks that encode information in the phases of spikes relative to the background oscillation. Our results show that firing one spike per cycle optimizes memory capacity in neurons encoding information in the phase of firing relative to a background rhythm.     

Principle of superposition-free memory

The requirement that memory be superposition-free imposes a restrictive condition on the structure of neural models. A simple model which satisfies this condition consists of two levels of neurons, with primary level neurons firing in response to external input and secondary (reference) neurons modifying activated primaries in such a way that they can later fire these primaries. Reference neurons are either activated by other reference neurons (time-ordered memories) or by primaries (content-ordered memories). The model allows for general powers of memory manipulation (e.g. formation of associative memories) through rememorization and has a number of implications which correspond to features of the brain.     

Reorganization of the human central nervous system

The key strategies on which the discovery of the functional organization of the central nervous system (CNS) under physiologic and pathophysiologic conditions have been based included (1) our measurements of phase and frequency coordination between the firings of alpha- and gamma-motoneurons and secondary muscle spindle afferents in the human spinal cord, (2) knowledge on CNS reorganization derived upon the improvement of the functions of the lesioned CNS in our patients in the short-term memory and the long-term memory (reorganization), and (3) the dynamic pattern approach for re-learning rhythmic coordinated behavior. The theory of self-organization and pattern formation in nonequilibrium systems is explicitly related to our measurements of the natural firing patterns of sets of identified single neurons in the human spinal premotor network and re-learned coordinated movements following spinal cord and brain lesions. Therapy induced cell proliferation, and maybe, neurogenesis seem to contribute to the host of structural changes during the process of re-learning of the lesioned CNS. So far, coordinated functions like movements could substantially be improved in every of the more than 100 patients with a CNS lesion by applying coordination dynamic therapy. As suggested by the data of our patients on re-learning, the human CNS seems to have a second integrative strategy for learning, re-learning, storing and recalling, which makes an essential contribution of the functional plasticity following a CNS lesion. A method has been developed by us for the simultaneous recording with wire electrodes of extracellular action potentials from single human afferent and efferent nerve fibres of undamaged sacral nerve roots. A classification scheme of the nerve fibres in the human peripheral nervous system (PNS) could be set up in which the individual classes of nerve fibres are characterized by group conduction velocities and group nerve fibre diameters. Natural impulse patterns of several identified single afferent and efferent nerve fibres (motoneuron axons) were extracted from multi-unit impulse patterns, and human CNS functions could be analyzed under physiologic and pathophysiologic conditions. With our discovery of premotor spinal oscillators it became possible to judge upon CNS neuronal network organization based on the firing patterns of these spinal oscillators and their driving afferents. Since motoneurons fire occasionally for low activation and oscillatory for high activation, the coherent organization of subnetworks to generate macroscopic function is very complex and for the time being, may be best described by the theory of coordination dynamics. Since oscillatory firing has also been observed by us in single motor unit firing patterns measured electromyographically, it seems possible to follow up therapeutic intervention in patients with spinal cord and brain lesions not only based on the activity levels and phases of motor programs during locomotion but also based on the physiologic and pathophysiologic firing patterns and recruitment of spinal oscillators. The improvement of the coordination dynamics of the CNS can be partly measured directly by rhythmicity upon the patient performing rhythmic movements coordinated up to milliseconds. Since rhythmic dynamic, coordinated, stereotyped movements are mainly located in the spinal cord and only little supraspinal drive is necessary to initiate, maintain, and terminate them, rhythmic, dynamic, coordinated movements were used in therapy to enforce reorganization of the lesioned CNS by improving the self-organization and relative coordination of spinal oscillators (and their interactions with occasionally firing motoneurons) which became pathologic in their firing following CNS lesion. Paraparetic, tetraparetic spinal cord and brain-lesioned patients re-learned running and other movements by an oscillator formation and coordination dynamic therapy. Our development in neurorehabilitation is in accordance with those of theoretical and computational neurosciences which deal with the self-organization of neuronal networks. In particular, jumping on a springboard 'in-phase' and in 'anti-phase' to re-learn phase relations of oscillator coupling can be understood in the framework of the Haken-Kelso-Bunz coordination dynamic model. By introducing broken symmetry, intention, learning and spasticity in the landscape of the potential function of the integrated CNS activity, the change in self-organization becomes understandable. Movement patterns re-learned by oscillator formation and coordination dynamic therapy evolve from reorganization and regeneration of the lesioned CNS by cooperative and competitive interplay between intrinsic coordination dynamics, extrinsic therapy related inputs with physiologic re-afferent input, including intention, motivation, supervised learning, interpersonal coordination, and genetic constraints including neurogenesis. (ABSTRACT TRUNCATED)     

Computational modeling of evolutionary learning processes in the brain

The evolutionary selection circuits model of learning has been specified algorithmically. The basic structural components of the selection circuits model are enzymatic neurons, that is, neurons whose firing behavior is controlled by membrane-bound macromolecules called excitases. Learning involves changes in the excitase contents of neurons through a process of variation and selection. In this paper we report on the behavior of a basic version of the learning algorithm which has been developed through extensive interactive experiments with the model. This algorithm is effective in that it enables single neurons or networks of neurons to learn simple pattern classification tasks in a number of time steps which appears experimentally to be a linear function of problem size, as measured by the number of patterns of presynaptic input. The experimental behavior of the algorithm establishes that evolutionary mechanisms of learning are competent to serve as major mechanisms of neuronal adaptation. As an example, we show how the evolutionary learning algorithm can contribute to adaptive motor control processes in which the learning system develops the ability to reach a target in the presence of randomly imposed disturbances.     

Emergence of network structure due to spike-timing-dependent plasticity in recurrent neuronal networks. II. Input selectivity—symmetry breaking

Spike-timing-dependent plasticity (STDP) is believed to structure neuronal networks by slowly changing the strengths (or weights) of the synaptic connections between neurons depending upon their spiking activity, which in turn modifies the neuronal firing dynamics. In this paper, we investigate the change in synaptic weights induced by STDP in a recurrently connected network in which the input weights are plastic but the recurrent weights are fixed. The inputs are divided into two pools with identical constant firing rates and equal within-pool spike-time correlations, but with no between-pool correlations. Our analysis uses the Poisson neuron model in order to predict the evolution of the input synaptic weights and focuses on the asymptotic weight distribution that emerges due to STDP. The learning dynamics induces a symmetry breaking for the individual neurons, namely for sufficiently strong within-pool spike-time correlation each neuron specializes to one of the input pools. We show that the presence of fixed excitatory recurrent connections between neurons induces a group symmetry-breaking effect, in which neurons tend to specialize to the same input pool. Consequently STDP generates a functional structure on the input connections of the network.     

Sequential configuration model for firing patterns in local neural networks

This paper presents a sequential configuration model to represent the coordinated firing patterns of memory traces in groups of neurons in local networks. Computer simulations are used to study the dynamic properties of memory traces selectively retrieved from networks in which multiple memory traces have been embedded according to the sequential configuration model. Distinct memory traces which utilize the same neurons, but differ only in temporal sequencing are selectively retrievable. Firing patterns of constituent neurons of retrieved memory traces exhibit the main properties of neurons observed in multi microelectrode recordings. The paper shows how to adjust relative synaptic weightings so as to control the disruptive influences of cross-talk in multipy-embedded networks. The theoretical distinction between (primarily anatomical) beds and (primarily physiological) realizations underlines the fundamentally stochastic nature of network firing patterns, and allows the definition of 4 degrees of clarity of retrieved memory traces.     

Role of microstructure of nerve impulse train in relation to transmission of neuronal activity and coding mechanism of neural information

Input-output relation were of giant neurons of a marine mollusc, Onchidium verruculatum, and a computer-simulated neuron investigated in terms of microstructure of nerve impulse train. The microstructure of input impulse train, the size of a unitary EPSP, and the extent of spontaneous firing activity of a single neuron had an important influence upon the effective summation of arriving synaptic inputs, the elicitation of output spikes, and intervals between succeeding output spikes. The neuron responded differently to respective input trains with different time structures, i.e. it discriminated input time pattern to various degrees. The manner in discrimination of input time pattern was dependent on the size of the unitary EPSP and the extent of the spontaneous firing activity, if it had. Some discussions were made with regard to possible coding systems of neural signal, assuming a frequency code and/or a pattern code.     

Firing rates of motoneurons with strong random synaptic excitation

The expected time to firing of a nerve impulse when there is Poisson excitation is calculated exactly in Stein's model. This is done at various input frequencies and various ratios of threshold to epsp magnitude, extending some previous calculations. The appropriate conditions for the validity of the model are discussed. Details of a particular calculation are given which involves the solution of a differential-difference equation. The results are presented as variation of expected time to firing as a function of input frequency for a given threshold to epsp ratio. The experimental results of Redman et al. for Poisson monosynaptic excitation of cat spinal motoneurons lead to the estimation of the epsp size which was not measured. The magnitude of the epsps predicted is in good agreement with that expected under the given conditions of stimulation. The predicted variation of epsp magnitude with input frequency is in accordance with that obtained in other experiments. When the finite rise time of epsps is taken into account the predicted epsp sizes are in better agreement with their expected amplitudes.     

Distributed neural networks of tactile working memory

Microelectrode recordings of cortical activity in primates performing working memory tasks reveal some cortical neurons exhibiting sustained or graded persistent elevations in firing rate during the period in which sensory information is actively maintained in short-term memory. These neurons are called “memory cells”. Imaging and transcranial magnetic stimulation studies indicate that memory cells may arise from distributed cortical networks. Depending on the sensory modality of the memorandum in working memory tasks, neurons exhibiting memory-correlated patterns of firing have been detected in different association cortices including prefrontal cortex, and primary sensory cortices as well.Here we elaborate on neurophysiological experiments that lead to our understanding of the neuromechanisms of working memory, and mainly discuss findings on widely distributed cortical networks involved in tactile working memory.     

Working Memory Cells' Behavior May Be Explained by Cross-Regional Networks with Synaptic Facilitation

Neurons in the cortex exhibit a number of patterns that correlate with working memory. Specifically, averaged across trials of working memory tasks, neurons exhibit different firing rate patterns during the delay of those tasks. These patterns include: 1) persistent fixed-frequency elevated rates above baseline, 2) elevated rates that decay throughout the tasks memory period, 3) rates that accelerate throughout the delay, and 4) patterns of inhibited firing (below baseline) analogous to each of the preceding excitatory patterns. Persistent elevated rate patterns are believed to be the neural correlate of working memory retention and preparation for execution of behavioral/motor responses as required in working memory tasks. Models have proposed that such activity corresponds to stable attractors in cortical neural networks with fixed synaptic weights. However, the variability in patterned behavior and the firing statistics of real neurons across the entire range of those behaviors across and within trials of working memory tasks are typical not reproduced. Here we examine the effect of dynamic synapses and network architectures with multiple cortical areas on the states and dynamics of working memory networks. The analysis indicates that the multiple pattern types exhibited by cells in working memory networks are inherent in networks with dynamic synapses, and that the variability and firing statistics in such networks with distributed architectures agree with that observed in the cortex.     

Optimal firing in sparsely-connected low-activity attractor networks

We examine the performance of Hebbian-like attractor neural networks, recalling stored memory patterns from their distorted versions. Searching for an activation (firing-rate) function that maximizes the performance in sparsely connected low-activity networks, we show that the optimal activation function is a threshold-sigmoid of the neuron's input field. This function is shown to be in close correspondence with the dependence of the firing rate of cortical neurons on their integrated input current, as described by neurophysiological recordings and conduction-based models. It also accounts for the decreasing-density shape of firing rates that has been reported in the literature. Received:9 December 1994 / Accepted in revised form: 9 January 1996     

Physical models of neural networks

The state of art in computer modelling of neural networks with associative memory is reviewed. The available experimental data are considered on learning and memory of small neural systems, on isolated synapses and on molecular level. Computer simulations demonstrate that realistic models of neural ensembles exhibit properties which can be interpreted as image recognition, categorization, learning, prototype forming, etc. A bilayer model of associative neural network is proposed. One layer corresponds to the short-term memory, the other one to the long-term memory. Patterns are stored in terms of the synaptic strength matrix. We have studied the relaxational dynamics of neurons firing and suppression within the short-term memory layer under the influence of the long-term memory layer. The interaction among the layers has found to create a number of novel stable states which are not the learning patterns. These synthetic patterns may consist of elements belonging to different non-intersecting learning patterns. Within the framework of a hypothesis of selective and definite coding of images in brain one can interpret the observed effect as the "idea? generating" process.     

Formation of feedforward networks and frequency synchrony by spike-timing-dependent plasticity

Spike-timing-dependent plasticity (STDP) with asymmetric learning windows is commonly found in the brain and useful for a variety of spike-based computations such as input filtering and associative memory. A natural consequence of STDP is establishment of causality in the sense that a neuron learns to fire with a lag after specific presynaptic neurons have fired. The effect of STDP on synchrony is elusive because spike synchrony implies unitary spike events of different neurons rather than a causal delayed relationship between neurons. We explore how synchrony can be facilitated by STDP in oscillator networks with a pacemaker. We show that STDP with asymmetric learning windows leads to self-organization of feedforward networks starting from the pacemaker. As a result, STDP drastically facilitates frequency synchrony. Even though differences in spike times are lessened as a result of synaptic plasticity, the finite time lag remains so that perfect spike synchrony is not realized. In contrast to traditional mechanisms of large-scale synchrony based on mutual interaction of coupled neurons, the route to synchrony discovered here is enslavement of downstream neurons by upstream ones. Facilitation of such feedforward synchrony does not occur for STDP with symmetric learning windows. Action Editor: Wulfram Gerstner     

Recurrent networks of submucous neurons controlling intestinal secretion: a modeling study

Secretomotor neurons, immunoreactive for vasoactive intestinal peptide (VIP), are important in controlling chloride secretion in the small intestine. These neurons form functional synapses with other submucosal VIP neurons and transmit via slow excitatory postsynaptic potentials (EPSPs). Thus they form a recurrent network with positive feedback. Intrinsic sensory neurons within the submucosa are also likely to form recurrent networks with positive feedback, provide substantial output to VIP neurons, and receive input from VIP neurons. If positive feedback within recurrent networks is sufficiently large, then neurons in the network respond to even small stimuli by firing at their maximum possible rate, even after the stimulus is removed. However, it is not clear whether such a mechanism operates within the recurrent networks of submucous neurons. We investigated this question by performing computer simulations of realistic models of VIP and intrinsic sensory neuron networks. In the expected range of electrophysiological properties, we found that activity in the VIP neuron network decayed slowly after cessation of a stimulus, indicating that positive feedback is not strong enough to support the uncontrolled firing state. The addition of intrinsic sensory neurons produced a low stable firing rate consistent with the common finding that basal secretory activity is, in part, neurogenic. Changing electrophysiological properties enables these recurrent networks to support the uncontrolled firing state, which may have implications with hypersecretion in the presence of enterotoxins such as cholera-toxin.     

A learning theory for reward-modulated spike-timing-dependent plasticity with application to biofeedback

Reward-modulated spike-timing-dependent plasticity (STDP) has recently emerged as a candidate for a learning rule that could explain how behaviorally relevant adaptive changes in complex networks of spiking neurons could be achieved in a self-organizing manner through local synaptic plasticity. However, the capabilities and limitations of this learning rule could so far only be tested through computer simulations. This article provides tools for an analytic treatment of reward-modulated STDP, which allows us to predict under which conditions reward-modulated STDP will achieve a desired learning effect. These analytical results imply that neurons can learn through reward-modulated STDP to classify not only spatial but also temporal firing patterns of presynaptic neurons. They also can learn to respond to specific presynaptic firing patterns with particular spike patterns. Finally, the resulting learning theory predicts that even difficult credit-assignment problems, where it is very hard to tell which synaptic weights should be modified in order to increase the global reward for the system, can be solved in a self-organizing manner through reward-modulated STDP. This yields an explanation for a fundamental experimental result on biofeedback in monkeys by Fetz and Baker. In this experiment monkeys were rewarded for increasing the firing rate of a particular neuron in the cortex and were able to solve this extremely difficult credit assignment problem. Our model for this experiment relies on a combination of reward-modulated STDP with variable spontaneous firing activity. Hence it also provides a possible functional explanation for trial-to-trial variability, which is characteristic for cortical networks of neurons but has no analogue in currently existing artificial computing systems. In addition our model demonstrates that reward-modulated STDP can be applied to all synapses in a large recurrent neural network without endangering the stability of the network dynamics.     

A neural network model based on the analogy with the immune system

The similarities between the immune system and the central nervous system lead to the formulation of an unorthodox neural network model. The similarities between the two systems are strong at the system level, but do not seem to be so striking at the level of the components. A new model of a neuron is therefore formulated, in order that the analogy can be used. The essential feature of the hypothetical neuron is that it exhibits hysteresis at the single neuron level. A network of N such neurons is modelled by an N-dimensional system of ordinary differential equations, which exhibits almost 2N attractors. The model has a property that resembles free will. A conjecture concerning how the network might learn stimulus-response behaviour is described. According to the conjecture, learning does not involve modifications of the strengths of synaptic connections. Instead, stimuli ("questions") selectively applied to the network by a "teacher" can be used to take the system to a region of the N-dimensional phase space where the network gives the desired stimulus-response behaviour. A key role for sleep in the learning process is suggested. The model for sleep leads to prediction that the variance in the rates of firing of the neurons associated with memory should increase during waking hours, and decrease during sleep.     

Learning invariant object recognition in the visual system with continuous transformations

The cerebral cortex utilizes spatiotemporal continuity in the world to help build invariant representations. In vision, these might be representations of objects. The temporal continuity typical of objects has been used in an associative learning rule with a short-term memory trace to help build invariant object representations. In this paper, we show that spatial continuity can also provide a basis for helping a system to self-organize invariant representations. We introduce a new learning paradigm “continuous transformation learning” which operates by mapping spatially similar input patterns to the same postsynaptic neurons in a competitive learning system. As the inputs move through the space of possible continuous transforms (e.g. translation, rotation, etc.), the active synapses are modified onto the set of postsynaptic neurons. Because other transforms of the same stimulus overlap with previously learned exemplars, a common set of postsynaptic neurons is activated by the new transforms, and learning of the new active inputs onto the same postsynaptic neurons is facilitated. We demonstrate that a hierarchical model of cortical processing in the ventral visual system can be trained with continuous transform learning, and highlight differences in the learning of invariant representations to those achieved by trace learning.     

Characterization of synchronized bursts in cultured hippocampal neuronal networks with learning training on microelectrode arrays

Spontaneous synchronized bursts seem to play a key role in brain functions such as learning and memory. Still controversial is the characterization of spontaneous synchronized bursts in neuronal networks after learning training, whether depression or promotion. By taking advantages of the main features of the microelectrode array (MEA) technology (i.e. multisite recordings, stable and long-term coupling with the biological preparation), we analyzed changes of spontaneous synchronized bursts in cultured hippocampal neuronal networks after learning training. And for this purpose, a learning model at networking level on MEA system was constructed, and analysis of spontaneous synchronized burst activity modulation was presented. Preliminary results show that, the number of burst was increased by 154%, burst duration was increased by 35%, and the number of spikes per burst was increased by 124%, while interburst interval decreased by 44% with learning. In particular, correlation and synchrony of neuronal activities in networks were enhanced by 51% and 36%, respectively, with learning. In contrast, dynamic properties of neuronal networks were not changed much when the network was under “non-learning” condition. These results indicate that firing, association and synchrony of spontaneous bursts in neuronal networks were promoted by learning. Furthermore, from these observations, we are encouraged to think of a more engineered system based on in vitro hippocampal neurons, as a novel sensitive system for electrophysiological evaluations.     

Information coding and oscillatory activity in synfire neural networks with and without inhibitory coupling

When a population spike (pulse-packet) propagates through a feedforward network with random excitatory connections, it either evolves to a sustained stable level of synchronous activity or fades away (Diesmann et al. in Nature 402:529-533 1999; Cateau and Fukai Neur Netw 14:675-685 2001). Here I demonstrate that in the presence of noise, the probability of the survival of the pulse-packet (or, equivalently, the firing rate of output neurons) reflects the intensity of the input. Furthermore, inhibitory coupling between layers can result in quasi- periodic alternation between several levels of firing activity. These results are obtained by analyzing the evolution of pulse-packet activity as a Markov chain. For the Markov chain analysis, the output of the chain is a linear mapping of the input into a lower-dimensional space, and the eigenvalues and eigenvectors of the transition matrix determine the dynamics of the evolution. Synchronous propagation of firing activity in successive pools of neurons are simulated in networks of integrate-and-fire and compartmental model neurons, and, consistent with the discrete Markov process, the activation of each pool is observed to be predominantly dependent upon the number of cells that fired in the previous pool. Simulation results agree with the numerical solutions of the Markov model. When inhibitory coupling between layers are included in the Markov model, some eigenvalues become complex numbers, implying oscillatory dynamics. The quasiperiodic dynamics is validated with simulation with leaky integrate-and-fire neurons. The networks demonstrate different modes of quasiperiodic activity as the inhibition or excitation parameters of the network are varied.