Adaptation to Delayed Force Perturbations in Reaching Movements

Adaptation to deterministic force perturbations during reaching movements was extensively studied in the last few decades. Here, we use this methodology to explore the ability of the brain to adapt to a delayed velocity-dependent force field. Two groups of subjects preformed a standard reaching experiment under a velocity dependent force field. The force was either immediately proportional to the current velocity (Control) or lagged it by 50 ms (Test). The results demonstrate clear adaptation to the delayed force perturbations. Deviations from a straight line during catch trials were shifted in time compared to post-adaptation to a non-delayed velocity dependent field (Control), indicating expectation to the delayed force field. Adaptation to force fields is considered to be a process in which the motor system predicts the forces to be expected based on the state that a limb will assume in response to motor commands. This study demonstrates for the first time that the temporal window of this prediction needs not to be fixed. This is relevant to the ability of the adaptive mechanisms to compensate for variability in the transmission of information across the sensory-motor system.     

Mechanisms of Motor Adaptation in Reactive Balance Control

Balance control must be rapidly modified to provide stability in the face of environmental challenges. Although changes in reactive balance over repeated perturbations have been observed previously, only anticipatory postural adjustments preceding voluntary movements have been studied in the framework of motor adaptation and learning theory. Here, we hypothesized that adaptation occurs in task-level balance control during responses to perturbations due to central changes in the control of both anticipatory and reactive components of balance. Our adaptation paradigm consisted of a Training set of forward support-surface perturbations, a Reversal set of novel countermanding perturbations that reversed direction, and a Washout set identical to the Training set. Adaptation was characterized by a change in a motor variable from the beginning to the end of each set, the presence of aftereffects at the beginning of the Washout set when the novel perturbations were removed, and a return of the variable at the end of the Washout to a level comparable to the end of the Training set. Task-level balance performance was characterized by peak center of mass (CoM) excursion and velocity, which showed adaptive changes with repetitive trials. Only small changes in anticipatory postural control, characterized by body lean and background muscle activity were observed. Adaptation was found in the evoked long-latency muscular response, and also in the sensorimotor transformation mediating that response. Finally, in each set, temporal patterns of muscle activity converged towards an optimum predicted by a trade-off between maximizing motor performance and minimizing muscle activity. Our results suggest that adaptation in balance, as well as other motor tasks, is mediated by altering central sensitivity to perturbations and may be driven by energetic considerations.     

Sensory motor coordination in an artificial gravity environment

The authors review and summarize research on the adaptation of limb movement control to Coriolis forces generated by body movements during rotation. They conclude that limb movement control can adapt to rotation rates as high as 10 rpm and that adaptation is rapid regardless of the presence or absence of visual and tactile feedback.     

Learning the dynamics of reaching movements results in the modification of arm impedance and long-latency perturbation responses

 Some characteristics of arm movements that humans exhibit during learning the dynamics of reaching are consistent with a theoretical framework where training results in motor commands that are gradually modified to predict and compensate for novel forces that may act on the hand. As a first approximation, the motor control system behaves as an adapting controller that learns an internal model of the dynamics of the task. It approximates inverse dynamics and predicts motor commands that are appropriate for a desired limb trajectory. However, we had previously noted that subtle motion characteristics observed during changes in task dynamics challenged this simple model and raised the possibility that adaptation also involved sensory–motor feedback pathways. These pathways reacted to sensory feedback during the course of the movement. Here we hypothesize that adaptation to dynamics might also involve a modification of how the CNS responds to sensory feedback. We tested this through experiments that quantified how the motor system's response to errors during voluntary movements changed as it adapted to dynamics of a force field. We describe a nonlinear approach that approximates the impedance of the arm, i.e., force response as a function of arm displacement trajectory. We observe that after adaptation, the impedance function changes in a way that closely matches and counters the effect of the force field. This is particularly prominent in the long-latency (>100 ms) component of response to perturbations. Therefore, it appears that practice not only modifies the internal model with which the brain generates motor commands that initiate a movement, but also the internal model with which sensory feedback is integrated with the ongoing descending commands in order to respond to error during the movement. Received: 10 January 2001 / Accepted in revised form: 30 May 2001     

Adapting to artificial gravity (AG) at high rotational speeds

Short-radius centrifugation offers a promising countermeasure to the adverse effects of prolonged weightlessness. Head movements made in a rotating environment elicit Coriolis effects, which seriously compromise sensory and motor processes. We have previously found that, contrary to common belief, participants can adapt to the Coriolis effects associated with single-quadrant yaw head turns during 23-rpm short-radius centrifugation, while maintaining their adaptation to stationary environments. Here, we focus on motion sickness and illusory motion, the most problematic subjective side effects. We present encouraging data that such context-specific adaptation generalizes immediately to a different centrifuge environment. It also generalizes quickly to Coriolis forces in the opposite direction. Implications for AG implementation are discussed.     

Adaptive dynamic programming as a theory of sensorimotor control

Many characteristics of sensorimotor control can be explained by models based on optimization and optimal control theories. However, most of the previous models assume that the central nervous system has access to the precise knowledge of the sensorimotor system and its interacting environment. This viewpoint is difficult to be justified theoretically and has not been convincingly validated by experiments. To address this problem, this paper presents a new computational mechanism for sensorimotor control from a perspective of adaptive dynamic programming (ADP), which shares some features of reinforcement learning. The ADP-based model for sensorimotor control suggests that a command signal for the human movement is derived directly from the real-time sensory data, without the need to identify the system dynamics. An iterative learning scheme based on the proposed ADP theory is developed, along with rigorous convergence analysis. Interestingly, the computational model as advocated here is able to reproduce the motor learning behavior observed in experiments where a divergent force field or velocity-dependent force field was present. In addition, this modeling strategy provides a clear way to perform stability analysis of the overall system. Hence, we conjecture that human sensorimotor systems use an ADP-type mechanism to control movements and to achieve successful adaptation to uncertainties present in the environment.     

Dynamic interactions between limb segments during planar arm movement

Movement of multiple segment limbs requires generation of appropriate joint torques which include terms arising from dynamic interactions among the moving segments as well as from such external forces as gravity. The interaction torques, arising from inertial, centripetal, and Coriolis forces, are not present for single joint movements. The significance of the individual interaction forces during reaching movements in a horizontal plane involving only the shoulder and elbow joints has been assessed for different movement paths and movement speeds. Trajectory formation strategies which simplify the dynamics computation are presented.     

Rethinking motor learning and savings in adaptation paradigms: model-free memory for successful actions combines with internal models

Although motor learning is likely to involve multiple processes, phenomena observed in error-based motor learning paradigms tend to be conceptualized in terms of only a single process: adaptation, which occurs through updating an internal model. Here we argue that fundamental phenomena like movement direction biases, savings (faster relearning), and interference do not relate to adaptation but instead are attributable to two additional learning processes that can be characterized as model-free: use-dependent plasticity and operant reinforcement. Although usually "hidden" behind adaptation, we demonstrate, with modified visuomotor rotation paradigms, that these distinct model-based and model-free processes combine to learn an error-based motor task. (1) Adaptation of an internal model channels movements toward successful error reduction in visual space. (2) Repetition of the newly adapted movement induces directional biases toward the?repeated movement. (3) Operant reinforcement through association of the adapted movement with successful error reduction is responsible for savings.     

Neural Tuning Functions Underlie Both Generalization and Interference

In sports, the role of backswing is considered critical for generating a good shot, even though it plays no direct role in hitting the ball. We recently demonstrated the scientific basis of this phenomenon by showing that immediate past movement affects the learning and recall of motor memories. This effect occurred regardless of whether the past contextual movement was performed actively, passively, or shown visually. In force field studies, it has been shown that motor memories generalize locally and that the level of compensation decays as a function of movement angle away from the trained movement. Here we examine if the contextual effect of past movement exhibits similar patterns of generalization and whether it can explain behavior seen in interference studies. Using a single force-field learning task, the directional tuning curves of both the prior contextual movement and the subsequent force field adaptive movements were measured. The adaptation movement direction showed strong directional tuning, decaying to zero by 90° relative to the training direction. The contextual movement direction exhibited a similar directional tuning, although the effect was always above 60%. We then investigated the directional tuning of the passive contextual movement using interference tasks, where the contextual movements that uniquely specified the force field direction were separated by ±15° or ±45°. Both groups showed a pronounced tuning effect, which could be well explained by the directional tuning functions for single force fields. Our results show that contextual effect of past movement influences predictive force compensation, even when adaptation does not require contextual information. However, when such past movement contextual information is crucial to the task, such as in an interference study, it plays a strong role in motor memory learning and recall. This work demonstrates that similar tuning responses underlie both generalization of movement direction during dynamic learning and contextual movements in both single force field and interference tasks.     

The binding of learning to action in motor adaptation

In motor tasks, errors between planned and actual movements generally result in adaptive changes which reduce the occurrence of similar errors in the future. It has commonly been assumed that the motor adaptation arising from an error occurring on a particular movement is specifically associated with the motion that was planned. Here we show that this is not the case. Instead, we demonstrate the binding of the adaptation arising from an error on a particular trial to the motion experienced on that same trial. The formation of this association means that future movements planned to resemble the motion experienced on a given trial benefit maximally from the adaptation arising from it. This reflects the idea that actual rather than planned motions are assigned 'credit' for motor errors because, in a computational sense, the maximal adaptive response would be associated with the condition credited with the error. We studied this process by examining the patterns of generalization associated with motor adaptation to novel dynamic environments during reaching arm movements in humans. We found that these patterns consistently matched those predicted by adaptation associated with the actual rather than the planned motion, with maximal generalization observed where actual motions were clustered. We followed up these findings by showing that a novel training procedure designed to leverage this newfound understanding of the binding of learning to action, can improve adaptation rates by greater than 50%. Our results provide a mechanistic framework for understanding the effects of partial assistance and error augmentation during neurologic rehabilitation, and they suggest ways to optimize their use.     

Correlations in state space can cause sub-optimal adaptation of optimal feedback control models

Control of our movements is apparently facilitated by an adaptive internal model in the cerebellum. It was long thought that this internal model implemented an adaptive inverse model and generated motor commands, but recently many reject that idea in favor of a forward model hypothesis. In theory, the forward model predicts upcoming state during reaching movements so the motor cortex can generate appropriate motor commands. Recent computational models of this process rely on the optimal feedback control (OFC) framework of control theory. OFC is a powerful tool for describing motor control, it does not describe adaptation. Some assume that adaptation of the forward model alone could explain motor adaptation, but this is widely understood to be overly simplistic. However, an adaptive optimal controller is difficult to implement. A reasonable alternative is to allow forward model adaptation to ‘re-tune’ the controller. Our simulations show that, as expected, forward model adaptation alone does not produce optimal trajectories during reaching movements perturbed by force fields. However, they also show that re-optimizing the controller from the forward model can be sub-optimal. This is because, in a system with state correlations or redundancies, accurate prediction requires different information than optimal control. We find that adding noise to the movements that matches noise found in human data is enough to overcome this problem. However, since the state space for control of real movements is far more complex than in our simple simulations, the effects of correlations on re-adaptation of the controller from the forward model cannot be overlooked.     

Concurrent adaptation of force and impedance in the redundant muscle system

This article examines the validity of a model to explain how humans learn to perform movements in environments with novel dynamics, including unstable dynamics typical of tool use. In this model, a simple rule specifies how the activation of each muscle is adapted from one movement to the next. Simulations of multijoint arm movements with a neuromuscular plant that incorporates neural delays, reflexes, and signal-dependent noise, demonstrate that the controller is able to compensate for changing internal or environment dynamics and noise properties. The computational model adapts by learning both the appropriate forces and required limb impedance to compensate precisely for forces and instabilities in arbitrary directions with patterns similar to those observed in motor learning experiments. It learns to regulate reciprocal activation and co-activation in a redundant muscle system during repeated movements without requiring any explicit transformation from hand to muscle space. Independent error-driven change in the activation of each muscle results in a coordinated control of the redundant muscle system and in a behavior that reduces instability, systematic error, and energy.     

Dynamics of Dual Prism Adaptation: Relating Novel Experimental Results to a Minimalistic Neural Model

In everyday life, humans interact with a dynamic environment often requiring rapid adaptation of visual perception and motor control. In particular, new visuo–motor mappings must be learned while old skills have to be kept, such that after adaptation, subjects may be able to quickly change between two different modes of generating movements (‘dual–adaptation’). A fundamental question is how the adaptation schedule determines the acquisition speed of new skills. Given a fixed number of movements in two different environments, will dual–adaptation be faster if switches (‘phase changes’) between the environments occur more frequently? We investigated the dynamics of dual–adaptation under different training schedules in a virtual pointing experiment. Surprisingly, we found that acquisition speed of dual visuo–motor mappings in a pointing task is largely independent of the number of phase changes. Next, we studied the neuronal mechanisms underlying this result and other key phenomena of dual–adaptation by relating model simulations to experimental data. We propose a simple and yet biologically plausible neural model consisting of a spatial mapping from an input layer to a pointing angle which is subjected to a global gain modulation. Adaptation is performed by reinforcement learning on the model parameters. Despite its simplicity, the model provides a unifying account for a broad range of experimental data: It quantitatively reproduced the learning rates in dual–adaptation experiments for both direct effect, i.e. adaptation to prisms, and aftereffect, i.e. behavior after removal of prisms, and their independence on the number of phase changes. Several other phenomena, e.g. initial pointing errors that are far smaller than the induced optical shift, were also captured. Moreover, the underlying mechanisms, a local adaptation of a spatial mapping and a global adaptation of a gain factor, explained asymmetric spatial transfer and generalization of prism adaptation, as observed in other experiments.     

Beyond Muscles Stiffness: Importance of State-Estimation to Account for Very Fast Motor Corrections

Feedback delays are a major challenge for any controlled process, and yet we are able to easily control limb movements with speed and grace. A popular hypothesis suggests that the brain largely mitigates the impact of feedback delays (∼50 ms) by regulating the limb intrinsic visco-elastic properties (or impedance) with muscle co-contraction, which generates forces proportional to changes in joint angle and velocity with zero delay. Although attractive, this hypothesis is often based on estimates of limb impedance that include neural feedback, and therefore describe the entire motor system. In addition, this approach does not systematically take into account that muscles exhibit high intrinsic impedance only for small perturbations (short-range impedance). As a consequence, it remains unclear how the nervous system handles large perturbations, as well as disturbances encountered during movement when short-range impedance cannot contribute. We address this issue by comparing feedback responses to load pulses applied to the elbow of human subjects with theoretical simulations. After validating the model parameters, we show that the ability of humans to generate fast and accurate corrective movements is compatible with a control strategy based on state estimation. We also highlight the merits of delay-uncompensated robust control, which can mitigate the impact of internal model errors, but at the cost of slowing feedback corrections. We speculate that the puzzling observation of presynaptic inhibition of peripheral afferents in the spinal cord at movement onset helps to counter the destabilizing transition from high muscle impedance during posture to low muscle impedance during movement.     

Computational nature of human adaptive control during learning of reaching movements in force fields

Learning to make reaching movements in force fields was used as a paradigm to explore the system architecture of the biological adaptive controller. We compared the performance of a number of candidate control systems that acted on a model of the neuromuscular system of the human arm and asked how well the dynamics of the candidate system compared with the movement characteristics of 16 subjects. We found that control via a supra-spinal system that utilized an adaptive inverse model resulted in dynamics that were similar to that observed in our subjects, but lacked essential characteristics. These characteristics pointed to a different architecture where descending commands were influenced by an adaptive forward model. However, we found that control via a forward model alone also resulted in dynamics that did not match the behavior of the human arm. We considered a third control architecture where a forward model was used in conjunction with an inverse model and found that the resulting dynamics were remarkably similar to that observed in the experimental data. The essential property of this control architecture was that it predicted a complex pattern of near-discontinuities in hand trajectory in the novel force field. A nearly identical pattern was observed in our subjects, suggesting that generation of descending motor commands was likely through a control system architecture that included both adaptive forward and inverse models. We found that as subjects learned to make reaching movements, adaptation rates for the forward and inverse models could be independently estimated and the resulting changes in performance of subjects from movement to movement could be accurately accounted for. Results suggested that the adaptation of the forward model played a dominant role in the motor learning of subjects. After a period of consolidation, the rates of adaptation in the internal models were significantly larger than those observed before the memory had consolidated. This suggested that consolidation of motor memory coincided with freeing of certain computational resources for subsequent learning. Received: 01 January 1998 / Accepted in revised form: 26 January 1999     

Visual feedback is not necessary for the learning of novel dynamics

Background

When learning to perform a novel sensorimotor task, humans integrate multi-modal sensory feedback such as vision and proprioception in order to make the appropriate adjustments to successfully complete the task. Sensory feedback is used both during movement to control and correct the current movement, and to update the feed-forward motor command for subsequent movements. Previous work has shown that adaptation to stable dynamics is possible without visual feedback. However, it is not clear to what degree visual information during movement contributes to this learning or whether it is essential to the development of an internal model or impedance controller.

Methodology/Principle Findings

We examined the effects of the removal of visual feedback during movement on the learning of both stable and unstable dynamics in comparison with the case when both vision and proprioception are available. Subjects were able to learn to make smooth movements in both types of novel dynamics after learning with or without visual feedback. By examining the endpoint stiffness and force after learning it could be shown that subjects adapted to both types of dynamics in the same way whether they were provided with visual feedback of their trajectory or not. The main effects of visual feedback were to increase the success rate of movements, slightly straighten the path, and significantly reduce variability near the end of the movement.

Conclusions/Significance

These findings suggest that visual feedback of the hand during movement is not necessary for the adaptation to either stable or unstable novel dynamics. Instead vision appears to be used to fine-tune corrections of hand trajectory at the end of reaching movements.     

Assessing the function of motor cortex: single-neuron models of how neural response is modulated by limb biomechanics

Do neurons in primary motor cortex encode the generative details of motor behavior, such as individual muscle activities, or do they encode high-level movement attributes? Resolving this question has proven difficult, in large part because of the sizeable uncertainty inherent in estimating or measuring the joint torques and muscle forces that underlie movements made by biological limbs. We circumvented this difficulty by considering single-neuron responses in an isometric task, where joint torques and muscle forces can be straightforwardly computed from limb geometry. The response for each neuron was modeled as a linear function of a "preferred" joint torque vector, and this model was fit to individual neural responses across variations in limb posture. The resulting goodness of fit suggests that neurons in motor cortex do encode the kinetics of motor behavior and that the neural response properties of "preferred direction" and "gain" are dual components of a unitary response vector.     

Remapping auditory-motor representations in voice production

Evidence regarding visually guided limb movements suggests that the motor system learns and maintains neural maps between motor commands and sensory feedback. Such systems are hypothesized to be used in a feed-forward control strategy that permits precision and stability without the delays of direct feedback control. Human vocalizations involve precise control over vocal and respiratory muscles. However, little is known about the sensorimotor representations underlying speech production. Here, we manipulated the heard fundamental frequency of the voice during speech to demonstrate learning of auditory-motor maps. Mandarin speakers repeatedly produced words with specific pitch patterns (tone categories). On each successive utterance, the frequency of their auditory feedback was increased by 1/100 of a semitone until they heard their feedback one full semitone above their true pitch. Subjects automatically compensated for these changes by lowering their vocal pitch. When feedback was unexpectedly returned to normal, speakers significantly increased the pitch of their productions beyond their initial baseline frequency. This adaptation was found to generalize to the production of another tone category. However, results indicate that a more robust adaptation was produced for the tone that was spoken during feedback alteration. The immediate aftereffects suggest a global remapping of the auditory-motor relationship after an extremely brief training period. However, this learning does not represent a complete transformation of the mapping; rather, it is in part target dependent.     

Reexposure to a sensorimotor perturbation produces opposite effects on explicit and implicit learning processes

The motor system demonstrates an exquisite ability to adapt to changes in the environment and to quickly reset when these changes prove transient. If similar environmental changes are encountered in the future, learning may be faster, a phenomenon known as savings. In studies of sensorimotor learning, a central component of savings is attributed to the explicit recall of the task structure and appropriate compensatory strategies. Whether implicit adaptation also contributes to savings remains subject to debate. We tackled this question by measuring, in parallel, explicit and implicit adaptive responses in a visuomotor rotation task, employing a protocol that typically elicits savings. While the initial rate of learning was faster in the second exposure to the perturbation, an analysis decomposing the 2 processes showed the benefit to be solely associated with explicit re-aiming. Surprisingly, we found a significant decrease after relearning in aftereffect magnitudes during no-feedback trials, a direct measure of implicit adaptation. In a second experiment, we isolated implicit adaptation using clamped visual feedback, a method known to eliminate the contribution of explicit learning processes. Consistent with the results of the first experiment, participants exhibited a marked reduction in the adaptation function, as well as an attenuated aftereffect when relearning from the clamped feedback. Motivated by these results, we reanalyzed data from prior studies and observed a consistent, yet unappreciated pattern of attenuation of implicit adaptation during relearning. These results indicate that explicit and implicit sensorimotor processes exhibit opposite effects upon relearning: Explicit learning shows savings, while implicit adaptation becomes attenuated

Humans learning a new motor task typically improve with repeated practice due to the faster expression of more effective explicit strategies; this study reveals that when motor learning occurs without awareness, performance deteriorates upon relearning.