Learning the optimal control of coordinated eye and head movements

Various optimality principles have been proposed to explain the characteristics of coordinated eye and head movements during visual orienting behavior. At the same time, researchers have suggested several neural models to underly the generation of saccades, but these do not include online learning as a mechanism of optimization. Here, we suggest an open-loop neural controller with a local adaptation mechanism that minimizes a proposed cost function. Simulations show that the characteristics of coordinated eye and head movements generated by this model match the experimental data in many aspects, including the relationship between amplitude, duration and peak velocity in head-restrained and the relative contribution of eye and head to the total gaze shift in head-free conditions. Our model is a first step towards bringing together an optimality principle and an incremental local learning mechanism into a unified control scheme for coordinated eye and head movements.     

Two-phase strategy of controlling motor coordination determined by task performance optimality

A quantitative model of optimal coordination between hand transport and grip aperture has been derived in our previous studies of reach-to-grasp movements without utilizing explicit knowledge of the optimality criterion or motor plant dynamics. The model’s utility for experimental data analysis has been demonstrated. Here we show how to generalize this model for a broad class of reaching-type, goal-directed movements. The model allows for measuring the variability of motor coordination and studying its dependence on movement phase. The experimentally found characteristics of that dependence imply that execution noise is low and does not affect motor coordination significantly. From those characteristics it is inferred that the cost of neural computations required for information acquisition and processing is included in the criterion of task performance optimality as a function of precision demand for state estimation and decision making. The precision demand is an additional optimized control variable that regulates the amount of neurocomputational resources activated dynamically. It is shown that an optimal control strategy in this case comprises two different phases. During the initial phase, the cost of neural computations is significantly reduced at the expense of reducing the demand for their precision, which results in speed-accuracy tradeoff violation and significant inter-trial variability of motor coordination. During the final phase, neural computations and thus motor coordination are considerably more precise to reduce the cost of errors in making a contact with the target object. The generality of the optimal coordination model and the two-phase control strategy is illustrated on several diverse examples.     

Control of rabbit nictitating membrane movements

Our objective in this study is to synthesize existing experimental data by constructing a realistic neuromechanical control model of rabbit nictitating membrane (NM) movements. We model the retractor bulbi muscle at the motor unit level because this is the level of nervous system control and also facilitates comparison with experimental data. Our motor unit model is derived from an earlier model of muscle activation based on calcium kinetics and includes a post-activation potentiation mechanism. Motor units are combined into a model of whole muscle that includes length-tension and force-velocity effects. Finally, we incorporate the muscle model into a biomechanical model in which the globe and NM are represented as a system of inertial, viscous, and elastic elements. The model takes patterns of neural signals (in the form of impulses) as input and produces movement of the NM as output. Our muscle model quantitatively accounts for data on isometric force development and decay for twitch, double shock, and tetanic stimulation. The complete model may be used for analysis of the relationship of motoneuron activity to behavior or as a realistic response generator in models of NM conditioning. This study also highlights gaps in the experimental data on the rabbit NM effector system.     

A model of the cerebellar pathways applied to the control of a single-joint robot arm actuated by McKibben artificial muscles

 This article describes an expanded version of a previously proposed motor control scheme, based on rules for combining sensory and motor signals within the central nervous system. Classical control elements of the previous cybernetic circuit were replaced by artificial neural network modules having an architecture based on the connectivity of the cerebellar cortex, and whose functioning is regulated by reinforcement learning. The resulting model was then applied to the motion control of a mechanical, single-joint robot arm actuated by two McKibben artificial muscles. Various biologically plausible learning schemes were studied using both simulations and experiments. After learning, the model was able to accurately pilot the movements of the robot arm, both in velocity and position. Received: 4 September 2000 / Accepted in revised form: 7 November 2001     

A hierarchical neural-network model for control and learning of voluntary movement

In order to control voluntary movements, the central nervous system (CNS) must solve the following three computational problems at different levels: the determination of a desired trajectory in the visual coordinates, the transformation of its coordinates to the body coordinates and the generation of motor command. Based on physiological knowledge and previous models, we propose a hierarchical neural network model which accounts for the generation of motor command. In our model the association cortex provides the motor cortex with the desired trajectory in the body coordinates, where the motor command is then calculated by means of long-loop sensory feedback. Within the spinocerebellum — magnocellular red nucleus system, an internal neural model of the dynamics of the musculoskeletal system is acquired with practice, because of the heterosynaptic plasticity, while monitoring the motor command and the results of movement. Internal feedback control with this dynamical model updates the motor command by predicting a possible error of movement. Within the cerebrocerebellum — parvocellular red nucleus system, an internal neural model of the inverse-dynamics of the musculo-skeletal system is acquired while monitoring the desired trajectory and the motor command. The inverse-dynamics model substitutes for other brain regions in the complex computation of the motor command. The dynamics and the inverse-dynamics models are realized by a parallel distributed neural network, which comprises many sub-systems computing various nonlinear transformations of input signals and a neuron with heterosynaptic plasticity (that is, changes of synaptic weights are assumed proportional to a product of two kinds of synaptic inputs). Control and learning performance of the model was investigated by computer simulation, in which a robotic manipulator was used as a controlled system, with the following results: (1) Both the dynamics and the inverse-dynamics models were acquired during control of movements. (2) As motor learning proceeded, the inverse-dynamics model gradually took the place of external feedback as the main controller. Concomitantly, overall control performance became much better. (3) Once the neural network model learned to control some movement, it could control quite different and faster movements. (4) The neural netowrk model worked well even when only very limited information about the fundamental dynamical structure of the controlled system was available. Consequently, the model not only accounts for the learning and control capability of the CNS, but also provides a promising parallel-distributed control scheme for a large-scale complex object whose dynamics are only partially known.     

An improved neural-network model for the neural integrator of the oculomotor system: More realistic neuron behavior

The discharge rates of premotor, brain-stem neurons that create eye movements modulate in relation to eye velocity yet firing rates of extraocular motoneurons contain both eye-position and eyevelocity signals. The eye-position signal is derived from the eye-velocity command by means of a neural network which functioins as a temporal integrator. We have previously proposed a network of lateral-inhibitory neurons that is capable of performing the required integration. That analysis centered on the temporal aspects of the signal processing for a limited class of idealized inputs. All of its cells were identical and carried only the integrated signal. Recordings in the brain stem, however, show that neurons in the region of the neural integrator have a variety of background firing rates, all carry some eye-velocity signal as well as the eye-position signal, and carry the former with different strengths depending on the type of eye movement being made. It was necessary to see if the proposed model could be modified to make its neurons more realistic.By modifying the spatial distribution of afferents to the network, we demonstrate that the same basic model functions properly in spite of afferents with nonuniform background firing rates. To introduce the eye-velocity signal a double-layer network, consisting of inhibitory and excitatory cells, was necessary. By presenting the velocity input to only local regions of this network it was shown that all cells in the network still carried the integrated signal and that its cells could carry different eye-velocity signals for different types of eye movements. Thus, this model stimulates quantitatively and qualitatively, the behavior of neurons seen in the region of the neural integrator.     

Movement curvature planning through force field internal models

Human motion studies have focused primarily on modeling straight point-to-point reaching movements. However, many goal-directed reaching movements, such as movements directed towards oneself, are not straight but rather follow highly curved trajectories. These movements are particularly interesting to study since they are essential in our everyday life, appear early in development and are routinely used to assess movement deficits following brain lesions. We argue that curved and straight-line reaching movements are generated by a unique neural controller and that the observed curvature of the movement is the result of an active control strategy that follows the geometry of one’s body, for instance to avoid trajectories that would hit the body or yield postures close to the joint limits. We present a mathematical model that accounts for such an active control strategy and show that the model reproduces with high accuracy the kinematic features of human data during unconstrained reaching movements directed toward the head. The model consists of a nonlinear dynamical system with a single stable attractor at the target. Embodiment-related task constraints are expressed as a force field that acts on the dynamical system. Finally, we discuss the biological plausibility and neural correlates of the model’s parameters and suggest that embodiment should be considered as a main cause for movement trajectory curvature.     

Grasping kinematics from the perspective of the individual digits: a modelling study

Grasping is a prototype of human motor coordination. Nevertheless, it is not known what determines the typical movement patterns of grasping. One way to approach this issue is by building models. We developed a model based on the movements of the individual digits. In our model the following objectives were taken into account for each digit: move smoothly to the preselected goal position on the object without hitting other surfaces, arrive at about the same time as the other digit and never move too far from the other digit. These objectives were implemented by regarding the tips of the digits as point masses with a spring between them, each attracted to its goal position and repelled from objects' surfaces. Their movements were damped. Using a single set of parameters, our model can reproduce a wider variety of experimental findings than any previous model of grasping. Apart from reproducing known effects (even the angles under which digits approach trapezoidal objects' surfaces, which no other model can explain), our model predicted that the increase in maximum grip aperture with object size should be greater for blocks than for cylinders. A survey of the literature shows that this is indeed how humans behave. The model can also adequately predict how single digit pointing movements are made. This supports the idea that grasping kinematics follow from the movements of the individual digits.     

Effect of grip strength and grip strengthening exercises on instantaneous bat velocity of collegiate baseball players

Bat velocity is considered to be an important factor for successful hitting. The relationship between grip strength and bat velocity has not been conclusively established. The purposes of this study were to determine the relationship of grip strength to bat velocity and to ascertain whether the performance of resistance training exercises designed to specifically target the forearms and grip would significantly alter bat velocity. The subjects for this study were 23 male members (mean +/- SD, age = 19.7 +/- 1.3 years, height = 182.5 +/- 5.9 cm, weight = 85.4 +/- 15.5 kg, experience = 14.4 +/- 1.7 years) of a varsity baseball team at a National Collegiate Athletic Association Division II school. The Jamar hand dynamometer was used to test grip strength, and the SETPRO Rookie was used to measure instantaneous bat velocity at the point of contact with the ball. Subjects were randomly divided into an experimental group and a control group. For 6 weeks, both groups participated in their usual baseball practice sessions, but the experimental group also performed extra forearm and grip strengthening exercises, whereas the control group did not. Pretest and posttest correlations between grip strength and bat velocity revealed no significant relationship between grip strength and bat velocity (pretest r = 0.054, p = 0.807; posttest r = 0.315, p = 0.145). A dependent t-test performed on all subjects revealed that a significant (p = 0.001) increase in bat velocity did occur over the course of the study. A covariate analysis, employing pretest bat velocity as the covariate, revealed no significant difference (p = 0.795) in posttest bat velocity scores between the experimental and control groups. Thus, increases in bat velocity occurred, but the differences were similar for both the experimental and control groups. The findings of this study suggest that grip strength and bat velocity are not significantly related, and that the allocation of time and energy for added training of the forearms in order to improve grip strength for the purpose of increasing bat velocity may not be warranted.     

Fast Feedback in Active Sensing: Touch-Induced Changes to Whisker-Object Interaction

Whisking mediated touch is an active sense whereby whisker movements are modulated by sensory input and behavioral context. Here we studied the effects of touching an object on whisking in head-fixed rats. Simultaneous movements of whiskers C1, C2, and D1 were tracked bilaterally and their movements compared. During free-air whisking, whisker protractions were typically characterized by a single acceleration-deceleration event, whisking amplitude and velocity were correlated, and whisk duration correlated with neither amplitude nor velocity. Upon contact with an object, a second acceleration-deceleration event occurred in about 25% of whisk cycles, involving both contacting (C2) and non-contacting (C1, D1) whiskers ipsilateral to the object. In these cases, the rostral whisker (C2) remained in contact with the object throughout the double-peak phase, which effectively prolonged the duration of C2 contact. These “touch-induced pumps” (TIPs) were detected, on average, 17.9 ms after contact. On a slower time scale, starting at the cycle following first touch, contralateral amplitude increased while ipsilateral amplitude decreased. Our results demonstrate that sensory-induced motor modulations occur at various timescales, and directly affect object palpation.     

Impedance characteristics of a neuromusculoskeletal model of the human arm II. Movement control

The modulation of neuromusculoskeletal impedance during movements is analysed using a motor control model of the human arm. The motor control system combines feedback and feedforward control and both control modes are determined in one optimization process. In the model, the stiffness varies at the double movement frequency for 2-Hz oscillatory elbow movements and has high values at the movement reversals. During goal-directed two-degrees-of-freedom arm movements, the stiffness is decreased during the movement and may be increased in the initial and final phases, depending on the movement velocity. The stiffness has a considerable curl during the movement, as was also observed in experimental data. The dynamic stiffness patterns of the model can be explained basically by the α−γ coactivation scheme where feedback gains covary with motor control signals. In addition to the modulation of the gain factors, it is argued that the variation of the intrinsic stiffness has a considerable effect on movement control, especially during fast movements. Received: 14 October 1997 / Accepted in revised form: 18 May 1999     

Feedback control in active sensing: rat exploratory whisking is modulated by environmental contact

Rats sweep their facial whiskers back and forth to generate tactile sensory information through contact with environmental structure. The neural processes operating on the signals arising from these whisker contacts are widely studied as a model of sensing in general, even though detailed knowledge of the natural circumstances under which such signals are generated is lacking. We used digital video tracking and wireless recording of mystacial electromyogram signals to assess the effects of whisker-object contact on whisking in freely moving animals exploring simple environments. Our results show that contact leads to reduced protraction (forward whisker motion) on the side of the animal ipsilateral to an obstruction and increased protraction on the contralateral side. Reduced ipsilateral protraction occurs rapidly and in the same whisk cycle as the initial contact. We conclude that whisker movements are actively controlled so as to increase the likelihood of environmental contacts while constraining such interactions to involve a gentle touch. That whisking pattern generation is under strong feedback control has important implications for understanding the nature of the signals reaching upstream neural processes.     

The generation of handwriting with delta-lognormal synergies

This paper presents a handwriting generation model that takes advantage of the asymptotic impulse response of neuromuscular networks to produce and control complex two-dimensional synergistic movements. A parametric definition of a ballistic stroke in the context of the kinematic theory of rapid human movements is given. Two types of parameters are used: command and system parameters. The first group provides a representation of the action plan while the second takes into account the temporal properties of the neuromuscular systems executing that plan. Handwriting is described as the time superimposition of basic discontinuous strokes that results in a continuous summation of delta-lognormal velocity vectors. The model leads to trajectory reconstruction, both in the spatial and in the kinematic domain. According to this new paradigm, the angular velocity does not have to be controlled independently and continuously; it naturally emerges from the vectorial summation process. Several psychophysical phenomena related to two-dimensional movements are explained and analyzed in the context of the model: the speed/accuracy trade-offs, spatial scaling, the isochrony principle, the two-thirds power law, effector independence, etc. The overall approach also shows how basic handwriting characteristics (dimension, slant, baseline, shape, etc.) are affected and controlled using an action plan made up of virtual targets fed into a neuromuscular synergy that is governed by a delta-lognormal law. Received: 22 July 1996 / Accepted in revised form: 15 September 1997     

Development of coordinated movement in chicks: II. Temporal analysis of hindlimb muscle synergies at embryonic day 10 in embryos with spinal gap transections.

Spinal neural circuits can recruit muscles to produce organized patterns of activity early in embryonic development. In a previous study, using multichannel electromyographic (EMG) recordings, we characterized burst parameters for these patterns in the legs of chick embryos during spontaneous motility in ovo at embryonic days (E) 9 and E10 (Bradley and Bekoff, 1990). Results of the study suggested both neural and biomechanical factors play an important role in the development of coordinated limb movements. In this study, to explore the contribution of descending neural inputs to the control of leg movements during motility, we applied similar methods to characterize motor patterns produced by the spinal cord in the absence of descending inputs. Thoracic spinal gap transections were performed at E2 and EMG patterns were recorded at E10. Several EMG features for chronic spinal embryos were similar to those for normal embryos and demonstrate that lumbar spinal circuits can be correctly assembled to control limb movements in the absence of connectivity with more rostral neural structures during early differentiation processes. However, certain aspects of the EMG patterns in chronic spinal embryos were different from patterns in normal embryos and provide support for conclusions drawn earlier by Oppenheim (1975). Specifically, our data support the view that propriospinal and/or supraspinal inputs function to regulate the timing of cyclic limb movements controlled by spinal neural circuits. Finally, we consider the possible long-term effects of chronic spinal gap transections as compared to acute spinal transections on the development of motility.     

Development of coordinated movement in chicks: II. Temporal analysis of hindlimb muscle synergies at embryonic day 10 in embryos with spinal gap transections

Spinal neural circuits can recruit muscles to produce organized patterns of activity early in embryonic development. In a previous study, using multichannel electromyographic (EMG) recordings, we characterized burst parameters for these patterns in the legs of chick embryos during spontaneous motility in ovo at embryonic days (E) 9 and E10 (Bradley and Bekoff, 1990). Results of the study suggested both neural and biomechanical factors play an important role in the development of coordinated limb movements. In this study, to explore the contribution of descending neural inputs to the control of leg movements during motility, we applied similar methods to characterize motor patterns produced by the spinal cord in the absence of descending inputs. Thoracic spinal gap transections were performed at E2 and EMG patterns were recorded at E10. Several EMG features for chronic spinal embryos were similar to those for normal embryos and demonstrate that lumbar spinal circuits can be correctly assembled to control limb movements in the absence of connectivity with more rostral neural structures during early differentiation processes. However, certain aspects of the EMG patterns in chronic spinal embryos were different from patterns in normal embryos and provide support for conclusions drawn earlier by Oppenheim (1975). Specifically, our data support the view that propriospinal and/or supraspinal inputs function to regulate the timing of cyclic limb movements controlled by spinal neural circuits. Finally, we consider the possible long-term effects of chronic spinal gap transections as compared to acute spinal transections on the development of motility. © 1992 John Wiley & Sons, Inc.     

The cerebellar nodulus/uvula integrates otolith signals for the translational vestibulo-ocular reflex

BACKGROUND: The otolith-driven translational vestibulo-ocular reflex (tVOR) generates compensatory eye movements to linear head accelerations. Studies in humans indicate that the cerebellum plays a critical role in the neural control of the tVOR, but little is known about mechanisms of this control or the functions of specific cerebellar structures. Here, we chose to investigate the contribution of the nodulus and uvula, which have been shown by prior studies to be involved in the processing of otolith signals in other contexts. METHODOLOGY/PRINCIPAL FINDINGS: We recorded eye movements in two rhesus monkeys during steps of linear motion along the interaural axis before and after surgical lesions of the cerebellar uvula and nodulus. The lesions strikingly reduced eye velocity during constant-velocity motion but had only a small effect on the response to initial head acceleration. We fit eye velocity to a linear combination of head acceleration and velocity and to a dynamic mathematical model of the tVOR that incorporated a specific integrator of head acceleration. Based on parameter optimization, the lesion decreased the gain of the pathway containing this new integrator by 62%. The component of eye velocity that depended directly on head acceleration changed little (gain decrease of 13%). In a final set of simulations, we compared our data to the predictions of previous models of the tVOR, none of which could account for our experimental findings. CONCLUSIONS/ SIGNIFICANCE: Our results provide new and important information regarding the neural control of the tVOR. Specifically, they point to a key role for the cerebellar nodulus and uvula in the mathematical integration of afferent linear head acceleration signals. This function is likely to be critical not only for the tVOR but also for the otolith-mediated reflexes that control posture and balance.     

Statistical decision theory and trade-offs in the control of motor response

We present a novel approach to the modeling of motor responses based on statistical decision theory. We begin with the hypothesis that subjects are ideal motion planners who choose movement trajectories to minimize expected loss. We derive predictions of the hypothesis for movement in environments where contact with specified regions carries rewards or penalties. The model predicts shifts in a subject's aiming point in response to changes in the reward and penalty structure of the environment and with changes in the subject's uncertainty in carrying out planned movements. We tested some of these predictions in an experiment where subjects were rewarded if they succeeded in touching a target region on a computer screen within a specified time limit. Near the target was a penalty region which, if touched, resulted in a penalty. We varied distance between the penalty region and the target and the cost of hitting the penalty region. Subjects shift their mean points of contact with the computer screen in response to changes in penalties and location of the penalty region relative to the target region in qualitative agreement with the predictions of the hypothesis. Thus, movement planning takes into account extrinsic costs and the subject's own motor uncertainty.     

Beyond Muscles Stiffness: Importance of State-Estimation to Account for Very Fast Motor Corrections

Feedback delays are a major challenge for any controlled process, and yet we are able to easily control limb movements with speed and grace. A popular hypothesis suggests that the brain largely mitigates the impact of feedback delays (∼50 ms) by regulating the limb intrinsic visco-elastic properties (or impedance) with muscle co-contraction, which generates forces proportional to changes in joint angle and velocity with zero delay. Although attractive, this hypothesis is often based on estimates of limb impedance that include neural feedback, and therefore describe the entire motor system. In addition, this approach does not systematically take into account that muscles exhibit high intrinsic impedance only for small perturbations (short-range impedance). As a consequence, it remains unclear how the nervous system handles large perturbations, as well as disturbances encountered during movement when short-range impedance cannot contribute. We address this issue by comparing feedback responses to load pulses applied to the elbow of human subjects with theoretical simulations. After validating the model parameters, we show that the ability of humans to generate fast and accurate corrective movements is compatible with a control strategy based on state estimation. We also highlight the merits of delay-uncompensated robust control, which can mitigate the impact of internal model errors, but at the cost of slowing feedback corrections. We speculate that the puzzling observation of presynaptic inhibition of peripheral afferents in the spinal cord at movement onset helps to counter the destabilizing transition from high muscle impedance during posture to low muscle impedance during movement.