Prediction precedes control in motor learning

Skilled motor behavior relies on the brain learning both to control the body and predict the consequences of this control. Prediction turns motor commands into expected sensory consequences, whereas control turns desired consequences into motor commands. To capture this symmetry, the neural processes underlying prediction and control are termed the forward and inverse internal models, respectively. Here, we investigate how these two fundamental processes are related during motor learning. We used an object manipulation task in which subjects learned to move a hand-held object with novel dynamic properties along a prescribed path. We independently and simultaneously measured subjects' ability to control their actions and to predict their consequences. We found different time courses for predictor and controller learning, with prediction being learned far more rapidly than control. In early stages of manipulating the object, subjects could predict the consequences of their actions, as measured by the grip force they used to grasp the object, but could not generate appropriate actions for control, as measured by their hand trajectory. As predicted by several recent theoretical models of sensorimotor control, our results indicate that people can learn to predict the consequences of their actions before they can learn to control their actions.     

Mantisbot is a robotic model of visually guided motion in the praying mantis

Insects use highly distributed nervous systems to process exteroception from head sensors, compare that information with state-based goals, and direct posture or locomotion toward those goals. To study how descending commands from brain centers produce coordinated, goal-directed motion in distributed nervous systems, we have constructed a conductance-based neural system for our robot MantisBot, a 29 degree-of-freedom, 13.3:1 scale praying mantis robot. Using the literature on mantis prey tracking and insect locomotion, we designed a hierarchical, distributed neural controller that establishes the goal, coordinates different joints, and executes prey-tracking motion. In our controller, brain networks perceive the location of prey and predict its future location, store this location in memory, and formulate descending commands for ballistic saccades like those seen in the animal. The descending commands are simple, indicating only 1) whether the robot should walk or stand still, and 2) the intended direction of motion. Each joint's controller uses the descending commands differently to alter sensory-motor interactions, changing the sensory pathways that coordinate the joints' central pattern generators into one cohesive motion. Experiments with one leg of MantisBot show that visual input produces simple descending commands that alter walking kinematics, change the walking direction in a predictable manner, enact reflex reversals when necessary, and can control both static posture and locomotion with the same network.     

Force and torque production in static multifinger prehension: biomechanics and control. II. Control

 The coordination of digits during combined force/torque production tasks was further studied using the data presented in the companion paper [Zatsiorsky et al. Biol Cybern this issue, Part I]. Optimization was performed using as criteria the cubic norms of (a) finger forces, (b) finger forces normalized with respect to the maximal forces measured in single-finger tasks, (c) finger forces normalized with respect to the maximal forces measured in a four-finger task, and (d) finger forces normalized with respect to the maximal moments that can be generated by the fingers. All four criteria failed to predict antagonist finger moments when these moments were not imposed by the task mechanics. Reconstruction of neural commands: The vector of neural commands c was reconstructed from the equation c=W ⁻¹ F, where W is the finger interconnection weight matrix and F is the vector of finger forces. The neural commands ranged from zero (no voluntary force production) to one (maximal voluntary contraction). For fingers producing moments counteracting the external torque (`agonist' fingers), the intensity of the neural commands was well correlated with the relative finger forces normalized to the maximal forces in a four-finger task. When fingers produced moments in the direction of the external torque (`antagonist' fingers), the relative finger forces were always larger than those expected from the intensity of the corresponding neural commands. The individual finger forces were decomposed into forces due to `direct' commands and forces induced by enslaving effects. Optimization of the neural commands resulted in the best correspondence between actual and predicted finger forces. The antagonist moments are, at least in part, due to enslaving effects: strong commands to agonist fingers also activated antagonist fingers. Received: 8 August 2001 / Accepted in revised form: 7 February 2002     

Learning the dynamics of reaching movements results in the modification of arm impedance and long-latency perturbation responses

 Some characteristics of arm movements that humans exhibit during learning the dynamics of reaching are consistent with a theoretical framework where training results in motor commands that are gradually modified to predict and compensate for novel forces that may act on the hand. As a first approximation, the motor control system behaves as an adapting controller that learns an internal model of the dynamics of the task. It approximates inverse dynamics and predicts motor commands that are appropriate for a desired limb trajectory. However, we had previously noted that subtle motion characteristics observed during changes in task dynamics challenged this simple model and raised the possibility that adaptation also involved sensory–motor feedback pathways. These pathways reacted to sensory feedback during the course of the movement. Here we hypothesize that adaptation to dynamics might also involve a modification of how the CNS responds to sensory feedback. We tested this through experiments that quantified how the motor system's response to errors during voluntary movements changed as it adapted to dynamics of a force field. We describe a nonlinear approach that approximates the impedance of the arm, i.e., force response as a function of arm displacement trajectory. We observe that after adaptation, the impedance function changes in a way that closely matches and counters the effect of the force field. This is particularly prominent in the long-latency (>100 ms) component of response to perturbations. Therefore, it appears that practice not only modifies the internal model with which the brain generates motor commands that initiate a movement, but also the internal model with which sensory feedback is integrated with the ongoing descending commands in order to respond to error during the movement. Received: 10 January 2001 / Accepted in revised form: 30 May 2001     

Learning from sensory and reward prediction errors during motor adaptation

Voluntary motor commands produce two kinds of consequences. Initially, a sensory consequence is observed in terms of activity in our primary sensory organs (e.g., vision, proprioception). Subsequently, the brain evaluates the sensory feedback and produces a subjective measure of utility or usefulness of the motor commands (e.g., reward). As a result, comparisons between predicted and observed consequences of motor commands produce two forms of prediction error. How do these errors contribute to changes in motor commands? Here, we considered a reach adaptation protocol and found that when high quality sensory feedback was available, adaptation of motor commands was driven almost exclusively by sensory prediction errors. This form of learning had a distinct signature: as motor commands adapted, the subjects altered their predictions regarding sensory consequences of motor commands, and generalized this learning broadly to neighboring motor commands. In contrast, as the quality of the sensory feedback degraded, adaptation of motor commands became more dependent on reward prediction errors. Reward prediction errors produced comparable changes in the motor commands, but produced no change in the predicted sensory consequences of motor commands, and generalized only locally. Because we found that there was a within subject correlation between generalization patterns and sensory remapping, it is plausible that during adaptation an individual''s relative reliance on sensory vs. reward prediction errors could be inferred. We suggest that while motor commands change because of sensory and reward prediction errors, only sensory prediction errors produce a change in the neural system that predicts sensory consequences of motor commands.     

Roles of dynamic and stationary components of a motor command in establishing the equilibrium state at simplest targeted movements

We studied in humans interrelations between the kinematic characteristics of targeted movements of the arm and current levels of EMG of the muscles providing these movements; the movements were relatively slow, and the attained joint angle was held for a time. The EMG level was considered a correlate of the level of integral motor commands (efferent activity of the respective motoneuronal pools). Application of low-amplitude non-inertial loadings, directed against the forces developed by one or another muscle group, allowed us to provide realization of targeted movements exclusively by the activity of this muscle group, without Involvement of the antagonists. It was demonstrated that the target equilibrium joint angle is reached synchronously with the dynamic phase of EMG activity, before the latter reaches a stationary level. The structure of the dynamic EMG phase itself is complex; in most cases it is split into several components. The dependence between the joint angle and amplitude of the EMG stationary phase is rather complex and variable, and usually it is difficult to predict the characteristics of this phase based on simple biomechanical considerations. There are proofs that at the performance of the studied movements and maintaining a target position there are some components in the mechanical muscle activity, which are not controlled by the motor commands. Thus, the stationary level of a motor command represents only one of several factors responsible for attaining and maintaining a target equilibrium position. Establishing this position is provided, first of all, by interaction of dynamic components of the motor commands to different muscles. Our results show that the attempts to interpret the processes of control of targeted movements on the basis of modifications of the equilibrium point hypothesis are inadequate; these data are in better compliance with the concept of impulse-temporal control; at their interpretation it is also necessary to take more thoroughly into account nonlinear properties of the muscle reactions.     

Adaptation to visual feedback delay influences visuomotor learning

Computational theory of motor control suggests that the brain continuously monitors motor commands, to predict their sensory consequences before actual sensory feedback becomes available. Such prediction error is a driving force of motor learning, and therefore appropriate associations between motor commands and delayed sensory feedback signals are crucial. Indeed, artificially introduced delays in visual feedback have been reported to degrade motor learning. However, considering our perceptual ability to causally bind our own actions with sensory feedback, demonstrated by the decrease in the perceived time delay following repeated exposure to an artificial delay, we hypothesized that such perceptual binding might alleviate deficits of motor learning associated with delayed visual feedback. Here, we evaluated this hypothesis by investigating the ability of human participants to adapt their reaching movements in response to a novel visuomotor environment with 3 visual feedback conditions--no-delay, sudden-delay, and adapted-delay. To introduce novelty into the trials, the cursor position, which originally indicated the hand position in baseline trials, was rotated around the starting position. In contrast to the no-delay condition, a 200-ms delay was artificially introduced between the cursor and hand positions during the presence of visual rotation (sudden-delay condition), or before the application of visual rotation (adapted-delay condition). We compared the learning rate (representing how the movement error modifies the movement direction in the subsequent trial) between the 3 conditions. In comparison with the no-delay condition, the learning rate was significantly degraded for the sudden-delay condition. However, this degradation was significantly alleviated by prior exposure to the delay (adapted-delay condition). Our data indicate the importance of appropriate temporal associations between motor commands and sensory feedback in visuomotor learning. Moreover, they suggest that the brain is able to account for such temporal associations in a flexible manner.     

Just do it: action-dependent learning allows sensory prediction

Sensory-motor learning is commonly considered as a mapping process, whereby sensory information is transformed into the motor commands that drive actions. However, this directional mapping, from inputs to outputs, is part of a loop; sensory stimuli cause actions and vice versa. Here, we explore whether actions affect the understanding of the sensory input that they cause. Using a visuo-motor task in humans, we demonstrate two types of learning-related behavioral effects. Stimulus-dependent effects reflect stimulus-response learning, while action-dependent effects reflect a distinct learning component, allowing the brain to predict the forthcoming sensory outcome of actions. Together, the stimulus-dependent and the action-dependent learning components allow the brain to construct a complete internal representation of the sensory-motor loop.     

Normalized Index of Synergy for Evaluating the Coordination of Motor Commands

Humans perform various motor tasks by coordinating the redundant motor elements in their bodies. The coordination of motor outputs is produced by motor commands, as well properties of the musculoskeletal system. The aim of this study was to dissociate the coordination of motor commands from motor outputs. First, we conducted simulation experiments where the total elbow torque was generated by a model of a simple human right and left elbow with redundant muscles. The results demonstrated that muscle tension with signal-dependent noise formed a coordinated structure of trial-to-trial variability of muscle tension. Therefore, the removal of signal-dependent noise effects was required to evaluate the coordination of motor commands. We proposed a method to evaluate the coordination of motor commands, which removed signal-dependent noise from the measured variability of muscle tension. We used uncontrolled manifold analysis to calculate a normalized index of synergy. Simulation experiments confirmed that the proposed method could appropriately represent the coordinated structure of the variability of motor commands. We also conducted experiments in which subjects performed the same task as in the simulation experiments. The normalized index of synergy revealed that the subjects coordinated their motor commands to achieve the task. Finally, the normalized index of synergy was applied to a motor learning task to determine the utility of the proposed method. We hypothesized that a large part of the change in the coordination of motor outputs through learning was because of changes in motor commands. In a motor learning task, subjects tracked a target trajectory of the total torque. The change in the coordination of muscle tension through learning was dominated by that of motor commands, which supported the hypothesis. We conclude that the normalized index of synergy can be used to evaluate the coordination of motor commands independently from the properties of the musculoskeletal system.     

Differences between Pacemaker and Nonpacemaker Neurons of Aplysia on Voltage Clamping

The responses of pacemaker and nonpacemaker Aplysia neurons to voltage clamp commands of less than 200 msec duration are essentially identical. For moderate depolarizing commands there is an early inward transient current followed by a late outward current and an outward tail current when the membrane is clamped back to resting potential. On long (1–2 sec) commands in pacemakers there is a marked sag in the late current and an inward tail current. E_tail, the potential of the membrane at which there is no net current flow under the conditions prevailing at the end of the clamp, shifts from about -9.0 mv on short commands to +5.0 mv on long commands. In contrast there is no marked sag of the late current or inward tail current on long commands in nonpacemakers, and E_tail is near -9.0 mv for both short and long commands. The current sag and shift in E_tail can be ascribed to a decreased conductance (presumably to K⁺) at the end of the long as compared to the short command in half of the pacemaker neurons. In the remaining cells the essential difference from nonpacemakers appears to be either a greater restricted extracellular space or a more active potential-dependent electrogenic Na⁺ pump in pacemakers.     

A biophysical approach to the Spatial Function of eye movements extraocular proprioception and the vestibulo-ocular reflex

The eyeball and the extraocular muscles are used as a paradigm to design a linear spatial model of a single joint with a redundant set of muscles. On the basis of this model relations are derived between orientation, torque, motor commands, and proprioceptive signals. These relations show that the tenet underlying the tensorial interpretation of neural signals in sensorimotor systems does not have general validity. A mechanism is proposed to show how proprioception may play a role in optimizing the coordination of muscles during spatial tasks. Further, a new concept is suggested that allows one to predict the neural connectivities mediating the redundant spatial vestibulo-ocular reflex. This concept has the advantage of minimizing both sensorial error and motor effort.     

A program system for interactive measurements on digitized cell images.

Using a high precision image scanner and a PDP-8/F minicomputer, we have developed a program system for interactive measurements on microscopic images. By giving simple keyboard commands, the operator can run the image scanner and manipulate the digitized images. The interface between the operator and the microscope-computer system is a Tektronix 4010 graphic terminal. The system allows objects to be isolated and parameters to be calculated from each object, e.g., parameters characterizing shape of the object, irregularity in light transmission over the object, area, integrated light transmission, etc. Objects are isolated and parameters are calculated under complete operator control using interactive computer graphics technique. Calculated parameters may be stored in dedicated data records, which are stored in files for later statistical analysis. The system also includes a statistical evaluation part. Technically, the system consists of a command scanner, which translates commands into internal representation, a parser, which checks the syntax of the commands, and an interpreter, which executes the commands. The system is designed so that new commands can be added easily.     

Central Activation of the Upper Limb Muscles in Humans Related to Creation of an Isometric Effort: Dependence on the Position of the Point of Force Application within the Operational Space

We examined the peculiarities of central coordination of motor commands coming to the muscles of the shoulder belt and shoulder in the course of generation of targeted isometric efforts by the arm. The dependence of these commands on changes in the effort direction and position of the forearm within the working space were analyzed. The intensity of the central commands was estimated according to the amplitudes of rectified and averaged EMGs recorded from the corresponding muscles. Sector diagrams of EMG activity of the above muscles depending on the direction of the effort vector, EV, were plotted [1]. Preferential sectors of activity where the efforts were formed due to activation of definite functional muscle groups were identified. As was found, the direction of these sectors depends significantly on the EV orientation. Differences between the patterns of coactivation of the examined muscles were demonstrated. Organization of the motor commands under conditions of creation of extensor efforts is distinguished by a more complex pattern than that related to flexor efforts. In the former case, the activity of extensor muscles is accompanied by more significant activation of the flexors.     

The influence of track compliance on running

A model of running is proposed in which the leg is represented as a rack-and-pinion element in series with a damped spring. The rack-and-pinion element emphasizes the role of descending commands, while the damped spring represents the dynamic properties of muscles and the position and the rate sensitivity of reflexes. This model is used to predict separately the effect of track compliance on step length and ground contact time. The predictions are compared with experiments in which athletes ran over tracks of controlled spring stiffness. A sharp spike in foot force up to 5 times body weight was found on hard surfaces, but this spike disappeared as the athletes ran on soft experimental tracks. Both ground contact time and step length increased on very compliant surfaces, leading to moderately reduced running speeds, but a range of track stiffness was discovered which actually enhances speed.     

Adaptive control of movement deceleration during saccades

As you read this text, your eyes make saccades that guide your fovea from one word to the next. Accuracy of these movements require the brain to monitor and learn from visual errors. A current model suggests that learning is supported by two different adaptive processes, one fast (high error sensitivity, low retention), and the other slow (low error sensitivity, high retention). Here, we searched for signatures of these hypothesized processes and found that following experience of a visual error, there was an adaptive change in the motor commands of the subsequent saccade. Surprisingly, this adaptation was not uniformly expressed throughout the movement. Rather, after experience of a single error, the adaptive response in the subsequent trial was limited to the deceleration period. After repeated exposure to the same error, the acceleration period commands also adapted, and exhibited resistance to forgetting during set-breaks. In contrast, the deceleration period commands adapted more rapidly, but suffered from poor retention during these same breaks. State-space models suggested that acceleration and deceleration periods were supported by a shared adaptive state which re-aimed the saccade, as well as two separate processes which resembled a two-state model: one that learned slowly and contributed primarily via acceleration period commands, and another that learned rapidly but contributed primarily via deceleration period commands.     

MOL3D,a modular and interactive program for molecular modeling and conformational analysis: I — basic modules

MOL3D is a generalized machine-independent computer program that lets the user interactively build 3D structures with different display options, such as wire, ball-and-stick and CPK representations. The program, which uses its own graphics package and driver, is designed to be very user friendly through the use of commands and menus. It has powerful transformation capabilities, such as software rotations, superpositions and zooming, and it is equipped with a fragment database that allows the user to build complex structures. The algorithm presented here is designed to perform computations in all the conformational space and therefore can be used to predict experimentally available quantities, such as NMR coupling constants. The program is efficient in the sense that it handles only dihedral angles in the first steps; as a result, it allows a rapid sampling of a great number of points through the entire conformational space. The user can choose between grid and Monte-Carlo searches of energy minimization, using a reasonable amount of computer time.     

Reduction of efferent motor commands during alpha feedback as a facilitator of EEG alpha and a precondition for changes in consciousness

Summary Evidence is presented in support of the hypothesis that a basic component underlies the subjective experience of the EEG alpha state. Corrective commands to the oculomotor system, causing movement in extrinsic eye muscles and lens adjustment, suppress occipital alpha. Somatic commands to many muscle groups block or suppress central alpha, hence the absence of such commands may be the precondition for an altered state of consciousness that can be repeatedly demonstrated by alpha feedback.Presented in part at the meeting of the Biofeedback Society, New Orleans, La., November 24–25, 1970.The author wishes to express his appreciation to Dr. Thomas B. Mulholland for his advice and suggestions.     

Gifa V. 4: A complete package for NMR data set processing

Summary The Gifa program is designed for processing, displaying and analysing 1D, 2D and 3D NMR data sets. It has been constructed in a modular fashion, based on three independent modules: a set of commands that perform all the basic processing operations such as apodisation functions, a complete set of Fourier Transforms, phasing and baseline correction, peak-picking and line fitting, linear prediction and maximum entropy processing; a set of command language primitives that permit the execution of complex macro commands; and a set of graphic commands that permit to build a complete graphic user interface, allowing the user to interact easily with the program. We have tried to create a versatile program that can be easily extended according to the user's requirements and that is adapted to a novice as well as an experienced user. The program runs on any UNIX computer, with or without graphic display, in interactive or batch mode.     

A neural network model of the vestibulo-ocular reflex using a local synaptic learning rule.

Vertebrates use the vestibulo-ocular reflex to maintain clear vision during head movements. This reflex requires eye-velocity commands from the semicircular canals to be integrated (mathematically) to produce eye-position commands for the extraocular muscles. This is accomplished by a neural network in the caudal pons. A model of this network is proposed using positive feedback via lateral inhibition. The model has been adapted to a learning network. We have developed a synaptic learning rule using only local information to make the model more physiological.     

Inspiratory resistances facilitate the diaphragm response to transcranial stimulation in humans

Background  

Breathing in humans is dually controlled for metabolic (brainstem commands) and behavioral purposes (suprapontine commands) with reciprocal modulation through spinal integration. Whereas the ventilatory response to chemical stimuli arises from the brainstem, the compensation of mechanical loads in awake humans is thought to involve suprapontine mechanisms. The aim of this study was to test this hypothesis by examining the effects of inspiratory resistive loading on the response of the diaphragm to transcranial magnetic stimulation.