A neuromorphic architecture for object recognition and motion anticipation using burst-STDP

In this work we investigate the possibilities offered by a minimal framework of artificial spiking neurons to be deployed in silico. Here we introduce a hierarchical network architecture of spiking neurons which learns to recognize moving objects in a visual environment and determine the correct motor output for each object. These tasks are learned through both supervised and unsupervised spike timing dependent plasticity (STDP). STDP is responsible for the strengthening (or weakening) of synapses in relation to pre- and post-synaptic spike times and has been described as a Hebbian paradigm taking place both in vitro and in vivo. We utilize a variation of STDP learning, called burst-STDP, which is based on the notion that, since spikes are expensive in terms of energy consumption, then strong bursting activity carries more information than single (sparse) spikes. Furthermore, this learning algorithm takes advantage of homeostatic renormalization, which has been hypothesized to promote memory consolidation during NREM sleep. Using this learning rule, we design a spiking neural network architecture capable of object recognition, motion detection, attention towards important objects, and motor control outputs. We demonstrate the abilities of our design in a simple environment with distractor objects, multiple objects moving concurrently, and in the presence of noise. Most importantly, we show how this neural network is capable of performing these tasks using a simple leaky-integrate-and-fire (LIF) neuron model with binary synapses, making it fully compatible with state-of-the-art digital neuromorphic hardware designs. As such, the building blocks and learning rules presented in this paper appear promising for scalable fully neuromorphic systems to be implemented in hardware chips.     

A real-time spiking cerebellum model for learning robot control

We describe a neural network model of the cerebellum based on integrate-and-fire spiking neurons with conductance-based synapses. The neuron characteristics are derived from our earlier detailed models of the different cerebellar neurons. We tested the cerebellum model in a real-time control application with a robotic platform. Delays were introduced in the different sensorimotor pathways according to the biological system. The main plasticity in the cerebellar model is a spike-timing dependent plasticity (STDP) at the parallel fiber to Purkinje cell connections. This STDP is driven by the inferior olive (IO) activity, which encodes an error signal using a novel probabilistic low frequency model. We demonstrate the cerebellar model in a robot control system using a target-reaching task. We test whether the system learns to reach different target positions in a non-destructive way, therefore abstracting a general dynamics model. To test the system's ability to self-adapt to different dynamical situations, we present results obtained after changing the dynamics of the robotic platform significantly (its friction and load). The experimental results show that the cerebellar-based system is able to adapt dynamically to different contexts.     

A computational mechanism for unified gain and timing control in the cerebellum

Precise gain and timing control is the goal of cerebellar motor learning. Because the basic neural circuitry of the cerebellum is homogeneous throughout the cerebellar cortex, a single computational mechanism may be used for simultaneous gain and timing control. Although many computational models of the cerebellum have been proposed for either gain or timing control, few models have aimed to unify them. In this paper, we hypothesize that gain and timing control can be unified by learning of the complete waveform of the desired movement profile instructed by climbing fiber signals. To justify our hypothesis, we adopted a large-scale spiking network model of the cerebellum, which was originally developed for cerebellar timing mechanisms to explain the experimental data of Pavlovian delay eyeblink conditioning, to the gain adaptation of optokinetic response (OKR) eye movements. By conducting large-scale computer simulations, we could reproduce some features of OKR adaptation, such as the learning-related change of simple spike firing of model Purkinje cells and vestibular nuclear neurons, simulated gain increase, and frequency-dependent gain increase. These results suggest that the cerebellum may use a single computational mechanism to control gain and timing simultaneously.     

A neural network model of the cerebellar cortex performing dynamic associations

The present paper proposes a model which applies formal neural network modeling techniques to construct a theoretical representation of the cerebellar cortex and its performances in motor control. A schema that makes explicit use of propagation delays of neural signals, is introduced to describe the ability to store temporal sequences of patterns in the Golgi-granule cell system. A perceptron association is then performed on these sequences of patterns by the Purkinje cell layer. The model conforms with important biological constraints, such as the known excitatory or inhibitory nature of the various synapses. Also, as suggested by experimental evidence, the synaptic plasticity underlying the learning ability of the model, is confined to the parallel fiber — Purkinje cell synapses, and takes place under the control of the climbing fibers. The result is a neural network model, constructed according to the anatomy of the cerebellar cortex, and capable of learning and retrieval of temporal sequences of patterns. It provides a framework to represent and interpret properties of learning and control of movements by the cerebellum, and to assess the capacity of formal neural network techniques for modeling of real neural systems.     

From sensors to spikes: evolving receptive fields to enhance sensorimotor information in a robot-arm

In biological systems, instead of actual encoders at different joints, proprioception signals are acquired through distributed receptive fields. In robotics, a single and accurate sensor output per link (encoder) is commonly used to track the position and the velocity. Interfacing bio-inspired control systems with spiking neural networks emulating the cerebellum with conventional robots is not a straight forward task. Therefore, it is necessary to adapt this one-dimensional measure (encoder output) into a multidimensional space (inputs for a spiking neural network) to connect, for instance, the spiking cerebellar architecture; i.e. a translation from an analog space into a distributed population coding in terms of spikes. This paper analyzes how evolved receptive fields (optimized towards information transmission) can efficiently generate a sensorimotor representation that facilitates its discrimination from other "sensorimotor states". This can be seen as an abstraction of the Cuneate Nucleus (CN) functionality in a robot-arm scenario. We model the CN as a spiking neuron population coding in time according to the response of mechanoreceptors during a multi-joint movement in a robot joint space. An encoding scheme that takes into account the relative spiking time of the signals propagating from peripheral nerve fibers to second-order somatosensory neurons is proposed. Due to the enormous number of possible encodings, we have applied an evolutionary algorithm to evolve the sensory receptive field representation from random to optimized encoding. Following the nature-inspired analogy, evolved configurations have shown to outperform simple hand-tuned configurations and other homogenized configurations based on the solution provided by the optimization engine (evolutionary algorithm). We have used artificial evolutionary engines as the optimization tool to circumvent nonlinearity responses in receptive fields.     

Tracking Plasticity: Effects of Long-Term Rehearsal in Expert Dancers Encoding Music to Movement

Our knowledge of neural plasticity suggests that neural networks show adaptation to environmental and intrinsic change. In particular, studies investigating the neuroplastic changes associated with learning and practicing motor tasks have shown that practicing such tasks results in an increase in neural activation in several specific brain regions. However, studies comparing experts and non-experts suggest that experts employ less neuronal activation than non-experts when performing a familiar motor task. Here, we aimed to determine the long-term changes in neural networks associated with learning a new dance in professional ballet dancers over 34 weeks. Subjects visualized dance movements to music while undergoing fMRI scanning at four time points over 34-weeks. Results demonstrated that initial learning and performance at seven weeks led to increases in activation in cortical regions during visualization compared to the first week. However, at 34 weeks, the cortical networks showed reduced activation compared to week seven. Specifically, motor learning and performance over the 34 weeks showed the typical inverted-U-shaped function of learning. Further, our result demonstrate that learning of a motor sequence of dance movements to music in the real world can be visualized by expert dancers using fMRI and capture highly significant modeled fits of the brain network variance of BOLD signals from early learning to expert level performance.     

Self-organization of spiking neural network that generates autonomous behavior in a real mobile robot

In this paper, we propose self-organization algorithm of spiking neural network (SNN) applicable to autonomous robot for generation of adoptive and goal-directed behavior. First, we formulated a SNN model whose inputs and outputs were analog and the hidden unites are interconnected each other. Next, we implemented it into a miniature mobile robot Khepera. In order to see whether or not a solution(s) for the given task(s) exists with the SNN, the robot was evolved with the genetic algorithm in the environment. The robot acquired the obstacle avoidance and navigation task successfully, exhibiting the presence of the solution. After that, a self-organization algorithm based on a use-dependent synaptic potentiation and depotentiation at synapses of input layer to hidden layer and of hidden layer to output layer was formulated and implemented into the robot. In the environment, the robot incrementally organized the network and the given tasks were successfully performed. The time needed to acquire the desired adoptive and goal-directed behavior using the proposed self-organization method was much less than that with the genetic evolution, approximately one fifth.     

Unsupervised learning of visual features through spike timing dependent plasticity

Spike timing dependent plasticity (STDP) is a learning rule that modifies synaptic strength as a function of the relative timing of pre- and postsynaptic spikes. When a neuron is repeatedly presented with similar inputs, STDP is known to have the effect of concentrating high synaptic weights on afferents that systematically fire early, while postsynaptic spike latencies decrease. Here we use this learning rule in an asynchronous feedforward spiking neural network that mimics the ventral visual pathway and shows that when the network is presented with natural images, selectivity to intermediate-complexity visual features emerges. Those features, which correspond to prototypical patterns that are both salient and consistently present in the images, are highly informative and enable robust object recognition, as demonstrated on various classification tasks. Taken together, these results show that temporal codes may be a key to understanding the phenomenal processing speed achieved by the visual system and that STDP can lead to fast and selective responses.     

Acute injections of brain-derived neurotrophic factor in a vocal premotor nucleus reversibly disrupt adult birdsong stability and trigger syllable deletion

Behavioral variability serves an essential role in motor learning by enabling sensory feedback to select those motor patterns that minimize error. Birds use auditory feedback to learn how to sing, and their songs lose variability and become highly stereotyped, or crystallized, at the end of a sensitive period for sensorimotor learning. The molecular cues that regulate song variability are not well understood. In other systems, neurotrophins, and brain-derived neurotrophic factor (BDNF) in particular, can mediate various forms of neural plasticity, including sensitive period neural circuit plasticity and activity-dependent synapse formation, and may also influence learning and memory. Here, we have tested the hypothesis that neurotrophin expression in the robust nucleus of the arcopallium (RA), the telencephalic output controlling song, regulates song variability. BDNF and its receptor trkB are expressed in RA, and BDNF expression in RA appears to be highest in juveniles, when song is most variable and plastic, and synapse density highest. Thus, song variability and synaptic connectivity could be enhanced by augmented expression of BDNF in RA. In support of this idea, we found that BDNF injections into the adult RA induced the re-expression of juvenile-like phenotypes, including song variability and an increased synaptic density in RA. Furthermore, BDNF treatment also induced vocal plasticity, characterized by syllable deletions and persistent changes to the song patterns. These results suggest that endogenous BDNF could be a molecular regulator of the song variability essential to vocal plasticity and, ultimately, to song learning.     

How the brain controls repetitive finger movements.

Adequate interaction with our physical and social environment requires accurate timing abilities. Since planning and control of movements is closely related to sensorimotor synchronization, the investigation of synchronization abilities may allow insights into fundamental principles of motor behaviour. The finger-tapping task has frequently been used to study the synchronization of one's own movements in relation to external events. Data from behavioural studies gave rise to the assumption that it is not the peripheral event (i.e., finger-tap or pacing signal) that is synchronized but its central representation. The neural foundations of sensorimotor synchronization have only recently been investigated and are still poorly understood. The present article reviews data from neurophysiological studies investigating sensorimotor synchronization to shed light on the neurophysiological processes associated with sensorimotor synchronization. This review focuses on studies investigating neuroelectric and neuromagnetic activity associated with simple repetitive synchronization tasks.     

Sensorimotor Learning Biases Choice Behavior: A Learning Neural Field Model for Decision Making

According to a prominent view of sensorimotor processing in primates, selection and specification of possible actions are not sequential operations. Rather, a decision for an action emerges from competition between different movement plans, which are specified and selected in parallel. For action choices which are based on ambiguous sensory input, the frontoparietal sensorimotor areas are considered part of the common underlying neural substrate for selection and specification of action. These areas have been shown capable of encoding alternative spatial motor goals in parallel during movement planning, and show signatures of competitive value-based selection among these goals. Since the same network is also involved in learning sensorimotor associations, competitive action selection (decision making) should not only be driven by the sensory evidence and expected reward in favor of either action, but also by the subject''s learning history of different sensorimotor associations. Previous computational models of competitive neural decision making used predefined associations between sensory input and corresponding motor output. Such hard-wiring does not allow modeling of how decisions are influenced by sensorimotor learning or by changing reward contingencies. We present a dynamic neural field model which learns arbitrary sensorimotor associations with a reward-driven Hebbian learning algorithm. We show that the model accurately simulates the dynamics of action selection with different reward contingencies, as observed in monkey cortical recordings, and that it correctly predicted the pattern of choice errors in a control experiment. With our adaptive model we demonstrate how network plasticity, which is required for association learning and adaptation to new reward contingencies, can influence choice behavior. The field model provides an integrated and dynamic account for the operations of sensorimotor integration, working memory and action selection required for decision making in ambiguous choice situations.     

Acute injections of brain‐derived neurotrophic factor in a vocal premotor nucleus reversibly disrupt adult birdsong stability and trigger syllable deletion

Behavioral variability serves an essential role in motor learning by enabling sensory feedback to select those motor patterns that minimize error. Birds use auditory feedback to learn how to sing, and their songs lose variability and become highly stereotyped, or crystallized, at the end of a sensitive period for sensorimotor learning. The molecular cues that regulate song variability are not well understood. In other systems, neurotrophins, and brain‐derived neurotrophic factor (BDNF) in particular, can mediate various forms of neural plasticity, including sensitive period neural circuit plasticity and activity‐dependent synapse formation, and may also influence learning and memory. Here, we have tested the hypothesis that neurotrophin expression in the robust nucleus of the arcopallium (RA), the telencephalic output controlling song, regulates song variability. BDNF and its receptor trkB are expressed in RA, and BDNF expression in RA appears to be highest in juveniles, when song is most variable and plastic, and synapse density highest. Thus, song variability and synaptic connectivity could be enhanced by augmented expression of BDNF in RA. In support of this idea, we found that BDNF injections into the adult RA induced the re‐expression of juvenile‐like phenotypes, including song variability and an increased synaptic density in RA. Furthermore, BDNF treatment also induced vocal plasticity, characterized by syllable deletions and persistent changes to the song patterns. These results suggest that endogenous BDNF could be a molecular regulator of the song variability essential to vocal plasticity and, ultimately, to song learning. © 2004 Wiley Periodicals, Inc. J Neurobiol, 2005     

A Real-Time Brain-Machine Interface Combining Motor Target and Trajectory Intent Using an Optimal Feedback Control Design

Real-time brain-machine interfaces (BMI) have focused on either estimating the continuous movement trajectory or target intent. However, natural movement often incorporates both. Additionally, BMIs can be modeled as a feedback control system in which the subject modulates the neural activity to move the prosthetic device towards a desired target while receiving real-time sensory feedback of the state of the movement. We develop a novel real-time BMI using an optimal feedback control design that jointly estimates the movement target and trajectory of monkeys in two stages. First, the target is decoded from neural spiking activity before movement initiation. Second, the trajectory is decoded by combining the decoded target with the peri-movement spiking activity using an optimal feedback control design. This design exploits a recursive Bayesian decoder that uses an optimal feedback control model of the sensorimotor system to take into account the intended target location and the sensory feedback in its trajectory estimation from spiking activity. The real-time BMI processes the spiking activity directly using point process modeling. We implement the BMI in experiments consisting of an instructed-delay center-out task in which monkeys are presented with a target location on the screen during a delay period and then have to move a cursor to it without touching the incorrect targets. We show that the two-stage BMI performs more accurately than either stage alone. Correct target prediction can compensate for inaccurate trajectory estimation and vice versa. The optimal feedback control design also results in trajectories that are smoother and have lower estimation error. The two-stage decoder also performs better than linear regression approaches in offline cross-validation analyses. Our results demonstrate the advantage of a BMI design that jointly estimates the target and trajectory of movement and more closely mimics the sensorimotor control system.     

Learning with slight forgetting optimizes sensorimotor transformation in redundant motor systems

Recent theoretical studies have proposed that the redundant motor system in humans achieves well-organized stereotypical movements by minimizing motor effort cost and motor error. However, it is unclear how this optimization process is implemented in the brain, presumably because conventional schemes have assumed a priori that the brain somehow constructs the optimal motor command, and largely ignored the underlying trial-by-trial learning process. In contrast, recent studies focusing on the trial-by-trial modification of motor commands based on error information suggested that forgetting (i.e., memory decay), which is usually considered as an inconvenient factor in motor learning, plays an important role in minimizing the motor effort cost. Here, we examine whether trial-by-trial error-feedback learning with slight forgetting could minimize the motor effort and error in a highly redundant neural network for sensorimotor transformation and whether it could predict the stereotypical activation patterns observed in primary motor cortex (M1) neurons. First, using a simple linear neural network model, we theoretically demonstrated that: 1) this algorithm consistently leads the neural network to converge at a unique optimal state; 2) the biomechanical properties of the musculoskeletal system necessarily determine the distribution of the preferred directions (PD; the direction in which the neuron is maximally active) of M1 neurons; and 3) the bias of the PDs is steadily formed during the minimization of the motor effort. Furthermore, using a non-linear network model with realistic musculoskeletal data, we demonstrated numerically that this algorithm could consistently reproduce the PD distribution observed in various motor tasks, including two-dimensional isometric torque production, two-dimensional reaching, and even three-dimensional reaching tasks. These results may suggest that slight forgetting in the sensorimotor transformation network is responsible for solving the redundancy problem in motor control.     

Lesions of the Basal forebrain cholinergic system impair task acquisition and abolish cortical plasticity associated with motor skill learning

The contribution of the basal forebrain cholinergic system in mediating plasticity of cortical sensorimotor representations was examined in the context of normal learning. The effects of specific basal forebrain cholinergic lesions upon cortical reorganization associated with learning a skilled motor task were investigated, addressing, for the first time, the functional consequences of blocking cortical map plasticity. Results demonstrate that disrupting basal forebrain cholinergic function disrupts cortical map reorganization and impairs motor learning. Cholinergic lesions do not impair associative fear learning or overall sensorimotor function. These results support the hypothesis that the basal forebrain cholinergic system may be specifically implicated in forms of learning requiring plasticity of cortical representations.     

Morphometric analysis of dendritic tree of cerebellar Purkinje-cells

Basic neural processes of sensorimotor adaptation can be observed by cellular-level studies on cerebellar cortex, both by electrophysiological and morphological means. In earlier studies we demonstrated double (sometimes triple or quadruple) rhythmic prespike activity patterns in dendritic microelectrode records taken from cerebellar Pc. By their active and passive interactions, the actual input pattern will turn into an arrhythmic output spiking, realizing a nonlinear and phase-sensitive integration. This curious complex spike-generating process can give rise to novel cerebellar functional models. The acting membrane dynamics rely not just upon the ionic current machinery but also upon the specific micromorphology of the dendritic tree, especially that of the branching areas. Although, statistical analyses on Pc dendrites generally followed the graph-theory, elaborated abstract parameters for complexity and hierarchy, and denied realistic geometry. Thus a novel specific morphometric study should be carried out, first applied to the main branching sites.     

Common features of neural activity during singing and sleep periods in a basal ganglia nucleus critical for vocal learning in a juvenile songbird

Reactivations of waking experiences during sleep have been considered fundamental neural processes for memory consolidation. In songbirds, evidence suggests the importance of sleep-related neuronal activity in song system motor pathway nuclei for both juvenile vocal learning and maintenance of adult song. Like those in singing motor nuclei, neurons in the basal ganglia nucleus Area X, part of the basal ganglia-thalamocortical circuit essential for vocal plasticity, exhibit singing-related activity. It is unclear, however, whether Area X neurons show any distinctive spiking activity during sleep similar to that during singing. Here we demonstrate that, during sleep, Area X pallidal neurons exhibit phasic spiking activity, which shares some firing properties with activity during singing. Shorter interspike intervals that almost exclusively occurred during singing in awake periods were also observed during sleep. The level of firing variability was consistently higher during singing and sleep than during awake non-singing states. Moreover, deceleration of firing rate, which is considered to be an important firing property for transmitting signals from Area X to the thalamic nucleus DLM, was observed mainly during sleep as well as during singing. These results suggest that songbird basal ganglia circuitry may be involved in the off-line processing potentially critical for vocal learning during sensorimotor learning phase.     

Implicit, predictive timing draws upon the same scalar representation of time as explicit timing

It is not yet known whether the scalar properties of explicit timing are also displayed by more implicit, predictive forms of timing. We investigated whether performance in both explicit and predictive timing tasks conformed to the two psychophysical properties of scalar timing: the Psychophysical law and Weber's law. Our explicit temporal generalization task required overt estimation of the duration of an empty interval bounded by visual markers, whereas our temporal expectancy task presented visual stimuli at temporally predictable intervals, which facilitated motor preparation thus speeding target detection. The Psychophysical Law and Weber's Law were modeled, respectively, by (1) the functional dependence between mean subjective time and real time (2) the linearity of the relationship between timing variability and duration. Results showed that performance for predictive, as well as explicit, timing conformed to both psychophysical properties of interval timing. Both tasks showed the same linear relationship between subjective and real time, demonstrating that the same representational mechanism is engaged whether it is transferred into an overt estimate of duration or used to optimise sensorimotor behavior. Moreover, variability increased with increasing duration during both tasks, consistent with a scalar representation of time in both predictive and explicit timing. However, timing variability was greater during predictive timing, at least for durations greater than 200 msec, and ascribable to temporal, rather than non-temporal, mechanisms engaged by the task. These results suggest that although the same internal representation of time was used in both tasks, its external manifestation varied as a function of temporal task goals.     

To spike,or when to spike?

Recent experimental reports have suggested that cortical networks can operate in regimes were sensory information is encoded by relatively small populations of spikes and their precise relative timing. Combined with the discovery of spike timing dependent plasticity, these findings have sparked growing interest in the capabilities of neurons to encode and decode spike timing based neural representations. To address these questions, a novel family of methodologically diverse supervised learning algorithms for spiking neuron models has been developed. These models have demonstrated the high capacity of simple neural architectures to operate also beyond the regime of the well established independent rate codes and to utilize theoretical advantages of spike timing as an additional coding dimension.