Is the movement representation in the motor cortex a moving target?

In redundant neural networks, many different combinations of connection weights will produce the same output, thereby providing many possible solutions for a given computation. In this issue of Neuron, Rokni et al. propose that the arm movement representations in the cerebral cortex act like redundant networks that drift randomly between different synaptic configurations with equivalent input-output behavior because of random noise in the adaptive learning mechanism.     

Using strategic movement to calibrate a neural compass: a spiking network for tracking head direction in rats and robots

The head direction (HD) system in mammals contains neurons that fire to represent the direction the animal is facing in its environment. The ability of these cells to reliably track head direction even after the removal of external sensory cues implies that the HD system is calibrated to function effectively using just internal (proprioceptive and vestibular) inputs. Rat pups and other infant mammals display stereotypical warm-up movements prior to locomotion in novel environments, and similar warm-up movements are seen in adult mammals with certain brain lesion-induced motor impairments. In this study we propose that synaptic learning mechanisms, in conjunction with appropriate movement strategies based on warm-up movements, can calibrate the HD system so that it functions effectively even in darkness. To examine the link between physical embodiment and neural control, and to determine that the system is robust to real-world phenomena, we implemented the synaptic mechanisms in a spiking neural network and tested it on a mobile robot platform. Results show that the combination of the synaptic learning mechanisms and warm-up movements are able to reliably calibrate the HD system so that it accurately tracks real-world head direction, and that calibration breaks down in systematic ways if certain movements are omitted. This work confirms that targeted, embodied behaviour can be used to calibrate neural systems, demonstrates that 'grounding' of modelled biological processes in the real world can reveal underlying functional principles (supporting the importance of robotics to biology), and proposes a functional role for stereotypical behaviours seen in infant mammals and those animals with certain motor deficits. We conjecture that these calibration principles may extend to the calibration of other neural systems involved in motion tracking and the representation of space, such as grid cells in entorhinal cortex.     

New experiences enhance coordinated neural activity in the hippocampus

The acquisition of new memories for places and events requires synaptic plasticity in the hippocampus, and plasticity depends on temporal coordination among neurons. Spatial activity in the hippocampus is relatively disorganized during the initial exploration of a novel environment, however, and it is unclear how neural activity during the initial stages of learning drives synaptic plasticity. Here we show that pairs of CA1 cells that represent overlapping novel locations are initially more coactive and more precisely coordinated than are cells representing overlapping familiar locations. This increased coordination occurs specifically during brief, high-frequency events (HFEs) in the local field potential that are similar to ripples and is not associated with better coordination of place-specific neural activity outside of HFEs. As novel locations become more familiar, correlations between cell pairs decrease. Thus, hippocampal neural activity during learning has a unique structure that is well suited to induce synaptic plasticity and to allow for rapid storage of new memories.     

Robust spatial working memory through homeostatic synaptic scaling in heterogeneous cortical networks

The concept of bell-shaped persistent neural activity represents a cornerstone of the theory for the internal representation of analog quantities, such as spatial location or head direction. Previous models, however, relied on the unrealistic assumption of network homogeneity. We investigate this issue in a network model where fine tuning of parameters is destroyed by heterogeneities in cellular and synaptic properties. Heterogeneities result in the loss of stored spatial information in a few seconds. Accurate encoding is recovered when a homeostatic mechanism scales the excitatory synapses to each cell to compensate for the heterogeneity in cellular excitability and synaptic inputs. Moreover, the more realistic model produces a wide diversity of tuning curves, as commonly observed in recordings from prefrontal neurons. We conclude that recurrent attractor networks in conjunction with appropriate homeostatic mechanisms provide a robust, biologically plausible theoretical framework for understanding the neural circuit basis of spatial working memory.     

Conserved fMRI and LFP signals during new associative learning in the human and macaque monkey medial temporal lobe

We measured local field potential (LFP) and blood-oxygen-level-dependent (BOLD) functional magnetic resonance imaging (fMRI) in the medial temporal lobes of monkeys and humans, respectively, as they performed the same conditional motor associative learning task. Parallel analyses were used to examine both data sets. Despite significantly faster learning in humans relative to monkeys, we found equivalent neural signals differentiating new versus highly familiar stimuli, first stimulus presentation, trial outcome, and learning strength in the entorhinal cortex and hippocampus of both species. Thus, the use of parallel behavioral tasks and analyses in monkeys and humans revealed conserved patterns of neural activity across the medial temporal lobe during an associative learning task.     

Attention modulation regulates both motor and non-motor performance: a high-density EEG study in Parkinson's disease

We have previously shown that, in early stages of Parkinson's disease (PD), patients with higher reaction times are also more impaired in visual sequence learning, suggesting that movement preparation shares resources with the learning of visuospatial sequences. Here, we ascertained whether, in patients with PD, the pattern of the neural correlates of attentional processes of movement planning predict sequence learning and working memory abilities. High density Electroencephalography (EEG, 256 electrodes) was recorded in 19 patients with PD performing reaching movements in a choice reaction time paradigm. Patients were also tested with Digit Span and performed a visuomotor sequence learning task that has an important declarative learning component. We found that attenuation of alpha/beta oscillatory activity before the stimulus presentation in frontoparietal regions significantly correlated with reaction time in the choice reaction time task, similarly to what we had previously found in normal subjects. In addition, such activity significantly predicted the declarative indices of sequence learning and the scores in the Digit Span task. These findings suggest that some motor and non motor PD signs might have common neural bases, and thus, might have a similar response to the same behavioral therapy. In addition, these results might help in designing and testing the efficacy of novel rehabilitative approaches to improve specific aspects of motor performance in PD and other neurological disorders.     

Emergence of symmetric, modular, and reciprocal connections in recurrent networks with Hebbian learning

While learning and development are well characterized in feedforward networks, these features are more difficult to analyze in recurrent networks due to the increased complexity of dual dynamics – the rapid dynamics arising from activation states and the slow dynamics arising from learning or developmental plasticity. We present analytical and numerical results that consider dual dynamics in a recurrent network undergoing Hebbian learning with either constant weight decay or weight normalization. Starting from initially random connections, the recurrent network develops symmetric or near-symmetric connections through Hebbian learning. Reciprocity and modularity arise naturally through correlations in the activation states. Additionally, weight normalization may be better than constant weight decay for the development of multiple attractor states that allow a diverse representation of the inputs. These results suggest a natural mechanism by which synaptic plasticity in recurrent networks such as cortical and brainstem premotor circuits could enhance neural computation and the generation of motor programs. Received: 27 April 1998 / Accepted in revised form: 16 March 1999     

Neural Tuning Functions Underlie Both Generalization and Interference

In sports, the role of backswing is considered critical for generating a good shot, even though it plays no direct role in hitting the ball. We recently demonstrated the scientific basis of this phenomenon by showing that immediate past movement affects the learning and recall of motor memories. This effect occurred regardless of whether the past contextual movement was performed actively, passively, or shown visually. In force field studies, it has been shown that motor memories generalize locally and that the level of compensation decays as a function of movement angle away from the trained movement. Here we examine if the contextual effect of past movement exhibits similar patterns of generalization and whether it can explain behavior seen in interference studies. Using a single force-field learning task, the directional tuning curves of both the prior contextual movement and the subsequent force field adaptive movements were measured. The adaptation movement direction showed strong directional tuning, decaying to zero by 90° relative to the training direction. The contextual movement direction exhibited a similar directional tuning, although the effect was always above 60%. We then investigated the directional tuning of the passive contextual movement using interference tasks, where the contextual movements that uniquely specified the force field direction were separated by ±15° or ±45°. Both groups showed a pronounced tuning effect, which could be well explained by the directional tuning functions for single force fields. Our results show that contextual effect of past movement influences predictive force compensation, even when adaptation does not require contextual information. However, when such past movement contextual information is crucial to the task, such as in an interference study, it plays a strong role in motor memory learning and recall. This work demonstrates that similar tuning responses underlie both generalization of movement direction during dynamic learning and contextual movements in both single force field and interference tasks.     

Encoding of luminance and contrast by linear and nonlinear synapses in the retina

Understanding how neural circuits transmit information is technically challenging because the neural code is contained in the activity of large numbers of neurons and synapses. Here, we use genetically encoded reporters to image synaptic transmission across a population of sensory neurons-bipolar cells in the retina of live zebrafish. We demonstrate that the luminance sensitivities of these synapses varies over 10(4) with a log-normal distribution. About half the synapses made by ON and OFF cells alter their polarity of transmission as a function of luminance to generate a triphasic tuning curve with distinct maxima and minima. These nonlinear synapses signal temporal contrast with greater sensitivity than linear ones. Triphasic tuning curves increase the dynamic range over which bipolar cells signal light and improve the efficiency with which luminance information is transmitted. The most efficient synapses signaled luminance using just 1 synaptic vesicle per second per distinguishable gray level.     

Representational drift as a window into neural and behavioural plasticity

Large-scale recordings of neural activity over days and weeks have revealed that neural representations of familiar tasks, precepts and actions continually evolve without obvious changes in behaviour. We hypothesise that this steady drift in neural activity and accompanying physiological changes is due in part to the continuous application of a learning rule at the cellular and population level. Explicit predictions of this drift can be found in neural network models that use iterative learning to optimise weights. Drift therefore provides a measurable signal that can reveal systems–level properties of biological plasticity mechanisms, such as their precision and effective learning rates.     

Tracking Plasticity: Effects of Long-Term Rehearsal in Expert Dancers Encoding Music to Movement

Our knowledge of neural plasticity suggests that neural networks show adaptation to environmental and intrinsic change. In particular, studies investigating the neuroplastic changes associated with learning and practicing motor tasks have shown that practicing such tasks results in an increase in neural activation in several specific brain regions. However, studies comparing experts and non-experts suggest that experts employ less neuronal activation than non-experts when performing a familiar motor task. Here, we aimed to determine the long-term changes in neural networks associated with learning a new dance in professional ballet dancers over 34 weeks. Subjects visualized dance movements to music while undergoing fMRI scanning at four time points over 34-weeks. Results demonstrated that initial learning and performance at seven weeks led to increases in activation in cortical regions during visualization compared to the first week. However, at 34 weeks, the cortical networks showed reduced activation compared to week seven. Specifically, motor learning and performance over the 34 weeks showed the typical inverted-U-shaped function of learning. Further, our result demonstrate that learning of a motor sequence of dance movements to music in the real world can be visualized by expert dancers using fMRI and capture highly significant modeled fits of the brain network variance of BOLD signals from early learning to expert level performance.     

Consolidation of dynamic motor learning is not disrupted by rTMS of primary motor cortex

Motor skills, once learned, are often retained over a long period of time. However, such learning first undergoes a period of consolidation after practice. During this time, the motor memory is susceptible to being disrupted by the performance of another motor-learning task. Recently, it was shown that repetitive transcranial magnetic stimulation (rTMS) over the primary motor cortex could disrupt the retention of a newly learned ballistic task in which subjects had to oppose their index finger and thumb as rapidly as possible. Here we investigate whether the motor cortex is similarly involved during the consolidation that follows learning novel dynamics. We applied rTMS to primary motor cortex shortly after subjects had either learned to compensate for a dynamic force field applied to their index finger or learned a ballistic finger abduction task. rTMS severely degraded the retention of the learning for the ballistic task but had no effect on retention of the dynamic force-field learning. This suggests that, unlike learning of simple ballistic skills, learning of dynamics may be stored in a more distributed manner, possibly outside the primary motor cortex.     

Interference in ballistic motor learning: specificity and role of sensory error signals

Humans are capable of learning numerous motor skills, but newly acquired skills may be abolished by subsequent learning. Here we ask what factors determine whether interference occurs in motor learning. We speculated that interference requires competing processes of synaptic plasticity in overlapping circuits and predicted specificity. To test this, subjects learned a ballistic motor task. Interference was observed following subsequent learning of an accuracy-tracking task, but only if the competing task involved the same muscles and movement direction. Interference was not observed from a non-learning task suggesting that interference requires competing learning. Subsequent learning of the competing task 4 h after initial learning did not cause interference suggesting disruption of early motor memory consolidation as one possible mechanism underlying interference. Repeated transcranial magnetic stimulation (rTMS) of corticospinal motor output at intensities below movement threshold did not cause interference, whereas suprathreshold rTMS evoking motor responses and (re)afferent activation did. Finally, the experiments revealed that suprathreshold repetitive electrical stimulation of the agonist (but not antagonist) peripheral nerve caused interference. The present study is, to our knowledge, the first to demonstrate that peripheral nerve stimulation may cause interference. The finding underscores the importance of sensory feedback as error signals in motor learning. We conclude that interference requires competing plasticity in overlapping circuits. Interference is remarkably specific for circuits involved in a specific movement and it may relate to sensory error signals.     

Prediction precedes control in motor learning

Skilled motor behavior relies on the brain learning both to control the body and predict the consequences of this control. Prediction turns motor commands into expected sensory consequences, whereas control turns desired consequences into motor commands. To capture this symmetry, the neural processes underlying prediction and control are termed the forward and inverse internal models, respectively. Here, we investigate how these two fundamental processes are related during motor learning. We used an object manipulation task in which subjects learned to move a hand-held object with novel dynamic properties along a prescribed path. We independently and simultaneously measured subjects' ability to control their actions and to predict their consequences. We found different time courses for predictor and controller learning, with prediction being learned far more rapidly than control. In early stages of manipulating the object, subjects could predict the consequences of their actions, as measured by the grip force they used to grasp the object, but could not generate appropriate actions for control, as measured by their hand trajectory. As predicted by several recent theoretical models of sensorimotor control, our results indicate that people can learn to predict the consequences of their actions before they can learn to control their actions.     

Nogo Receptor 1 Limits Tactile Task Performance Independent of Basal Anatomical Plasticity

The genes that govern how experience refines neural circuitry and alters synaptic structural plasticity are poorly understood. The nogo-66 receptor 1 gene (ngr1) is one candidate that may restrict the rate of learning as well as basal anatomical plasticity in adult cerebral cortex. To investigate if ngr1 limits the rate of learning we tested adult ngr1 null mice on a tactile learning task. Ngr1 mutants display greater overall performance despite a normal rate of improvement on the gap-cross assay, a whisker-dependent learning paradigm. To determine if ngr1 restricts basal anatomical plasticity in the associated sensory cortex, we repeatedly imaged dendritic spines and axonal varicosities of both constitutive and conditional adult ngr1 mutant mice in somatosensory barrel cortex for two weeks through cranial windows with two-photon chronic in vivo imaging. Neither constant nor acute deletion of ngr1 affected turnover or stability of dendritic spines or axonal boutons. The improved performance on the gap-cross task is not attributable to greater motor coordination, as ngr1 mutant mice possess a mild deficit in overall performance and a normal learning rate on the rotarod, a motor task. Mice lacking ngr1 also exhibit normal induction of tone-associated fear conditioning yet accelerated fear extinction and impaired consolidation. Thus, ngr1 alters tactile and motor task performance but does not appear to limit the rate of tactile or motor learning, nor determine the low set point for synaptic turnover in sensory cortex.     

Dopamine Promotes Motor Cortex Plasticity and Motor Skill Learning via PLC Activation

Dopaminergic neurons in the ventral tegmental area, the major midbrain nucleus projecting to the motor cortex, play a key role in motor skill learning and motor cortex synaptic plasticity. Dopamine D1 and D2 receptor antagonists exert parallel effects in the motor system: they impair motor skill learning and reduce long-term potentiation. Traditionally, D1 and D2 receptor modulate adenylyl cyclase activity and cyclic adenosine monophosphate accumulation in opposite directions via different G-proteins and bidirectionally modulate protein kinase A (PKA), leading to distinct physiological and behavioral effects. Here we show that D1 and D2 receptor activity influences motor skill acquisition and long term synaptic potentiation via phospholipase C (PLC) activation in rat primary motor cortex. Learning a new forelimb reaching task is severely impaired in the presence of PLC, but not PKA-inhibitor. Similarly, long term potentiation in motor cortex, a mechanism involved in motor skill learning, is reduced when PLC is inhibited but remains unaffected by the PKA inhibitor. Skill learning deficits and reduced synaptic plasticity caused by dopamine antagonists are prevented by co-administration of a PLC agonist. These results provide evidence for a role of intracellular PLC signaling in motor skill learning and associated cortical synaptic plasticity, challenging the traditional view of bidirectional modulation of PKA by D1 and D2 receptors. These findings reveal a novel and important action of dopamine in motor cortex that might be a future target for selective therapeutic interventions to support learning and recovery of movement resulting from injury and disease.     

Accuracy in estimating motor commands using neuronal population coding

We consider the problem of estimating motor commands from an ensemble of neuronal activities. The population vector algorithm proposed by Georgopoulos provides largely biased estimations when preferred directions of neurons are non-uniformly distributed. To improve this, various decoding methods have been proposed. However, dependence of decoding accuracy on the motor command and other features of neural activities, such as baseline firing rates or amplitudes of tuning curves, have not been quantitatively discussed. In this study, we propose a new method to estimate the motor command in the maximum likelihood estimation framework, which is analytically tractable. We find that the estimation accuracy is independent of the motor command. Using our estimation method, we can estimate the motor command with equal accuracy in all directions.     

Action selection and action value in frontal-striatal circuits

The role that frontal-striatal circuits play in normal behavior remains unclear. Two of the leading hypotheses suggest that these circuits are important for action selection or reinforcement learning. To examine these hypotheses, we carried out an experiment in which monkeys had to select actions in two different task conditions. In the first (random) condition, actions were selected on the basis of perceptual inference. In the second (fixed) condition, the animals used reinforcement from previous trials to select actions. Examination of neural activity showed that the representation of the selected action was stronger in lateral prefrontal cortex (lPFC), and occurred earlier in the lPFC than it did in the dorsal striatum (dSTR). In contrast to this, the representation of action values, in both the random and fixed conditions, was stronger in the dSTR. Thus, the dSTR contains an enriched representation of action value, but it followed frontal cortex in action selection.     

Neocortical mechanisms in motor learning

The ability to learn novel motor skills has fundamental importance for adaptive behavior. Neocortical mechanisms support human motor skill learning, from simple practice to adaptation and arbitrary sensory-motor associations. Behavioral and neural manifestations of motor learning evolve in time and involve multiple structures across the neocortex. Modifications of neural properties, synchrony and synaptic efficacy are all related to the development and maintenance of motor skill.     

A symbiotic brain-machine interface through value-based decision making

Background

In the development of Brain Machine Interfaces (BMIs), there is a great need to enable users to interact with changing environments during the activities of daily life. It is expected that the number and scope of the learning tasks encountered during interaction with the environment as well as the pattern of brain activity will vary over time. These conditions, in addition to neural reorganization, pose a challenge to decoding neural commands for BMIs. We have developed a new BMI framework in which a computational agent symbiotically decoded users'' intended actions by utilizing both motor commands and goal information directly from the brain through a continuous Perception-Action-Reward Cycle (PARC).

Methodology

The control architecture designed was based on Actor-Critic learning, which is a PARC-based reinforcement learning method. Our neurophysiology studies in rat models suggested that Nucleus Accumbens (NAcc) contained a rich representation of goal information in terms of predicting the probability of earning reward and it could be translated into an evaluative feedback for adaptation of the decoder with high precision. Simulated neural control experiments showed that the system was able to maintain high performance in decoding neural motor commands during novel tasks or in the presence of reorganization in the neural input. We then implanted a dual micro-wire array in the primary motor cortex (M1) and the NAcc of rat brain and implemented a full closed-loop system in which robot actions were decoded from the single unit activity in M1 based on an evaluative feedback that was estimated from NAcc.

Conclusions

Our results suggest that adapting the BMI decoder with an evaluative feedback that is directly extracted from the brain is a possible solution to the problem of operating BMIs in changing environments with dynamic neural signals. During closed-loop control, the agent was able to solve a reaching task by capturing the action and reward interdependency in the brain.