Evidence for composite cost functions in arm movement planning: an inverse optimal control approach

An important issue in motor control is understanding the basic principles underlying the accomplishment of natural movements. According to optimal control theory, the problem can be stated in these terms: what cost function do we optimize to coordinate the many more degrees of freedom than necessary to fulfill a specific motor goal? This question has not received a final answer yet, since what is optimized partly depends on the requirements of the task. Many cost functions were proposed in the past, and most of them were found to be in agreement with experimental data. Therefore, the actual principles on which the brain relies to achieve a certain motor behavior are still unclear. Existing results might suggest that movements are not the results of the minimization of single but rather of composite cost functions. In order to better clarify this last point, we consider an innovative experimental paradigm characterized by arm reaching with target redundancy. Within this framework, we make use of an inverse optimal control technique to automatically infer the (combination of) optimality criteria that best fit the experimental data. Results show that the subjects exhibited a consistent behavior during each experimental condition, even though the target point was not prescribed in advance. Inverse and direct optimal control together reveal that the average arm trajectories were best replicated when optimizing the combination of two cost functions, nominally a mix between the absolute work of torques and the integrated squared joint acceleration. Our results thus support the cost combination hypothesis and demonstrate that the recorded movements were closely linked to the combination of two complementary functions related to mechanical energy expenditure and joint-level smoothness.     

Learning the optimal control of coordinated eye and head movements

Various optimality principles have been proposed to explain the characteristics of coordinated eye and head movements during visual orienting behavior. At the same time, researchers have suggested several neural models to underly the generation of saccades, but these do not include online learning as a mechanism of optimization. Here, we suggest an open-loop neural controller with a local adaptation mechanism that minimizes a proposed cost function. Simulations show that the characteristics of coordinated eye and head movements generated by this model match the experimental data in many aspects, including the relationship between amplitude, duration and peak velocity in head-restrained and the relative contribution of eye and head to the total gaze shift in head-free conditions. Our model is a first step towards bringing together an optimality principle and an incremental local learning mechanism into a unified control scheme for coordinated eye and head movements.     

Kinematic invariants during cyclical arm movements

It has been observed that the motion of the arm end-point (the hand, fingertip or the tip of a pen) is characterized by a number of regularities (kinematic invariants). Trajectory is usually straight, and the velocity profile has a bell shape during point-to-point movements. During drawing movements, a two-thirds power law predicts the dependence of the end-point velocity on the trajectory curvature. Although various principles of movement organization have been discussed as possible origins of these kinematic invariants, the nature of these movement trajectory characteristics remains an open question. A kinematic model of cyclical arm movements derived in the present study analytically demonstrates that all three kinematic invariants can be predicted from a two-joint approximation of the kinematic structure of the arm and from sinusoidal joint motions. With this approach, explicit expressions for two kinematic invariants, the two-thirds power law during drawing movements and the velocity profile during point-to-point movements are obtained as functions of arm segment lengths and joint motion parameters. Additionally, less recognized kinematic invariants are also derived from the model. The obtained analytical expressions are further validated with experimental data. The high accuracy of the predictions confirms practical utility of the model, showing that the model is relevant to human performance over a wide range of movements. The results create a basis for the consolidation of various existing interpretations of kinematic invariants. In particular, optimal control is discussed as a plausible source of invariant characteristics of joint motions and movement trajectories.     

Two-phase strategy of controlling motor coordination determined by task performance optimality

A quantitative model of optimal coordination between hand transport and grip aperture has been derived in our previous studies of reach-to-grasp movements without utilizing explicit knowledge of the optimality criterion or motor plant dynamics. The model’s utility for experimental data analysis has been demonstrated. Here we show how to generalize this model for a broad class of reaching-type, goal-directed movements. The model allows for measuring the variability of motor coordination and studying its dependence on movement phase. The experimentally found characteristics of that dependence imply that execution noise is low and does not affect motor coordination significantly. From those characteristics it is inferred that the cost of neural computations required for information acquisition and processing is included in the criterion of task performance optimality as a function of precision demand for state estimation and decision making. The precision demand is an additional optimized control variable that regulates the amount of neurocomputational resources activated dynamically. It is shown that an optimal control strategy in this case comprises two different phases. During the initial phase, the cost of neural computations is significantly reduced at the expense of reducing the demand for their precision, which results in speed-accuracy tradeoff violation and significant inter-trial variability of motor coordination. During the final phase, neural computations and thus motor coordination are considerably more precise to reduce the cost of errors in making a contact with the target object. The generality of the optimal coordination model and the two-phase control strategy is illustrated on several diverse examples.     

Analysis of an optimal control model of multi-joint arm movements

 In this paper, we propose a model of biological motor control for generation of goal-directed multi-joint arm movements, and study the formation of muscle control inputs and invariant kinematic features of movements. The model has a hierarchical structure that can determine the control inputs for a set of redundant muscles without any inverse computation. Calculation of motor commands is divided into two stages, each of which performs a transformation of motor commands from one coordinate system to another. At the first level, a central controller in the brain accepts instructions from higher centers, which represent the motor goal in the Cartesian space. The controller computes joint equilibrium trajectories and excitation signals according to a minimum effort criterion. At the second level, a neural network in the spinal cord translates the excitation signals and equilibrium trajectories into control commands to three pairs of antagonist muscles which are redundant for a two-joint arm. No inverse computation is required in the determination of individual muscle commands. The minimum effort controller can produce arm movements whose dynamic and kinematic features are similar to those of voluntary arm movements. For fast movements, the hand approaches a target position along a near-straight path with a smooth bell-shaped velocity. The equilibrium trajectories in X and Y show an ‘N’ shape, but the end-point equilibrium path zigzags around the hand path. Joint movements are not always smooth. Joint reversal is found in movements in some directions. The excitation signals have a triphasic (or biphasic) pulse pattern, which leads to stereotyped triphasic (or biphasic) bursts in muscle control inputs, and a dynamically modulated joint stiffness. There is a fixed sequence of muscle activation from proximal muscles to distal muscles. The order is preserved in all movements. For slow movements, it is shown that a constant joint stiffness is necessary to produce a smooth movement with a bell-shaped velocity. Scaled movements can be reproduced by varying the constraints on the maximal level of excitation signals according to the speed of movement. When the inertial parameters of the arm are altered, movement trajectories can be kept invariant by adjusting the pulse height values, showing the ability to adapt to load changes. These results agree with a wide range of experimental observations on human voluntary movements. Received: 4 December 1995 / Accepted in revised form: 17 September 1996     

Velocity-Based Planning of Rapid Elbow Movements Expands the Control Scheme of the Equilibrium Point Hypothesis

According to the equilibrium point hypothesis of voluntary motor control, control action of muscles is not explicitly computed, but rather arises as a consequence of interaction between moving equilibrium position, current kinematics and stiffness of the joint. This approach is attractive as it obviates the need to explicitly specify the forces controlling limb movements. However, many debatable aspects of this hypothesis remain in the manner of specification of the equilibrium point trajectory and muscle activation (or its stiffness), which elicits a restoring force toward the planned equilibrium trajectory. In this study, we expanded the framework of this hypothesis by assuming that the control system uses the velocity measure as the origin of subordinate variables scaling descending commands. The velocity command is translated into muscle control inputs by second order pattern generators, which yield reciprocal command and coactivation commands, and create alternating activation of the antagonistic muscles during movement and coactivation in the post-movement phase, respectively. The velocity command is also integrated to give a position command specifying a moving equilibrium point. This model is purely kinematics-dependent, since the descending commands needed to modulate the visco-elasticity of muscles are implicitly given by simple parametric specifications of the velocity command alone. The simulated movements of fast elbow single-joint movements corresponded well with measured data performed over a wide range of movement distances, in terms of both muscle excitations and kinematics. Our proposal on a synthesis for the equilibrium point approach and velocity command, may offer some insights into the control scheme of the single-joint arm movements.     

Optimization and evaluation of a proportional derivative controller for planar arm movement

In most clinical applications of functional electrical stimulation (FES), the timing and amplitude of electrical stimuli have been controlled by open-loop pattern generators. The control of upper extremity reaching movements, however, will require feedback control to achieve the required precision. Here we present three controllers using proportional derivative (PD) feedback to stimulate six arm muscles, using two joint angle sensors. Controllers were first optimized and then evaluated on a computational arm model that includes musculoskeletal dynamics. Feedback gains were optimized by minimizing a weighted sum of position errors and muscle forces. Generalizability of the controllers was evaluated by performing movements for which the controller was not optimized, and robustness was tested via model simulations with randomly weakened muscles. Robustness was further evaluated by adding joint friction and doubling the arm mass. After optimization with a properly weighted cost function, all PD controllers performed fast, accurate, and robust reaching movements in simulation. Oscillatory behavior was seen after improper tuning. Performance improved slightly as the complexity of the feedback gain matrix increased.     

Analysis of a muscular control system in human movements

In this work, we have studied a muscular control system under experimental conditions for analyzing the dynamic behavior of individual muscles and theoretical considerations for elucidating its control strategy. Movement of human limbs is achieved by joint torques and each torque is specified as the sum of torques generated by muscle forces. The behavior of individual muscles is controlled by the neural input which is estimated by means of an electromyogram (EMG). In this study, the EMGs for a flexor and an extensor are measured in elbow joint movements and the dynamic behavior of individual muscles is analyzed. As a result, it is verified that both a flexor and an extensor are activated throughout the entire movement and that the activation of muscles is controlled above a specific limit independent of the hand-held load. Subsequently, a system model for simulating elbow joint movements is developed which includes the muscle dynamic relationship between the neural input and the isometric force. The minimum limit of muscle activation that has been confirmed in experiments is provided as a constraint of the neural input and the criterion is defined by a derivative of the isometric force of individual muscles. The optimal trajectories formulated under these conditions are quantitatively compared with the experimentally observed trajectories, and the control strategy of a muscular control system is studied. Finally, a muscular control system in multi-joint arm movements is discussed with regard to the comparative analysis between observed and optimal trajectories. Received: 7 April 1999 / Accepted in revised form: 27 July 1999     

Trajectory formation based on physiological characteristics of skeletal muscles

Human arm trajectories in natural unrestricted reaching movements were studied. They have particular properties such that a hand path is a rather simple straight or curved line, and a tangential velocity profile of hand is bell-shaped. Also these properties are invariant, independent of movement duration and hand-held load. In this study, trajectory formation is investigated on the basis of physiological characteristics of skeletal muscles, and a criterion prescribed by a derivative of isometric muscle torque is proposed. Subsequently, optimal trajectories are formulated under various conditions of movement to account for a planning strategy of human arm trajectories. In addition to such a theoretical approach, human arm trajectories are experimentally observed by a measuring system which provides a visual sensor and a target tracking device, enabling totally unrestricted movements. Then, optimal trajectories are quantitatively evaluated in comparison with experimental data in which essential properties of human arm trajectories are demonstrated. These results support the idea that human arm trajectories are planned in order to minimize the proposed criterion which is determined from physiological aspects. Finally, the physiological advantages of human arm trajectories are discussed with regard to the analysis of observed and optimal trajectories. Received: 2 December 1997 / Accepted in revised form: 20 March 1998     

A theory of optimal differential gene expression

We investigate a model of optimal regulation, intended to describe large-scale differential gene expression. Relations between the optimal expression patterns and the function of genes are deduced from an optimality principle: the regulators have to maximise a fitness function which they influence directly via a cost term, and indirectly via their control on important cell variables, such as metabolic fluxes. According to the model, the optimal linear response to small perturbations reflects the regulators' functions, namely their linear influences on the cell variables. The optimal behaviour can be realised by a linear feedback mechanism. Known or assumed properties of response coefficients lead to predictions about regulation patterns. A symmetry relation predicted for deletion experiments is verified with gene expression data. Where the optimality assumption is valid, our results justify the use of expression data for functional annotation and for pathway reconstruction and suggest the use of linear factor models for the analysis of gene expression data.     

Virtual trajectory and stiffness ellipse during multijoint arm movement predicted by neural inverse models

We predict the virtual trajectories and stiffness ellipses during multijoint arm movements by computer simulations. A two-link manipulator with four single-joint muscles and two double-joint muscles is used as a model of the human arm. Physical parameters of the model are derived from several experimental data. Among them, special emphasis is put on low values of the dynamic hand stiffness recently measured during single joint and multijoint movements. The feedback-error-learning scheme to acquire the inverse dynamics model and the inverse statics model is utilized for this prediction. The virtual trajectories are much more complex than the actual trajectories. This indicates that planning the virtual trajectory is as difficult as solving the inverse dynamics problem for medium and fast movements, and simply falsifies the advocated computational advantage of the virtual trajectory control hypothesis. Thus, we conclude that learning inverse models is essential even in the virtual trajectory control framework. Finally, we propose a new computational model to learn the complicated shape of the virtual trajectories by integrating the virtual trajectory control and the feedback-error-learning scheme.     

Optimal control of antagonistic muscle stiffness during voluntary movements

This paper presents a study on the control of antagonist muscle stiffness during single-joint arm movements by optimal control theory with a minimal effort criterion. A hierarchical model is developed based on the physiology of the neuromuscular control system and the equilibrium point hypothesis. For point-to-point movements, the model provides predictions on (1) movement trajectory, (2) equilibrium trajectory, (3) muscle control inputs, and (4) antagonist muscle stiffness, as well as other variables. We compared these model predictions to the behavior observed in normal human subjects. The optimal movements capture the major invariant characteristics of voluntary movements, such as a sigmoidal movement trajectory with a bell-shaped velocity profile, an N-shaped equilibrium trajectory, a triphasic burst pattern of muscle control inputs, and a dynamically modulated joint stiffness. The joint stiffness is found to increase in the middle of the movement as a consequence of the triphasic muscle activities. We have also investigated the effects of changes in model parameters on movement control. We found that the movement kinematics and muscle control inputs are strongly influenced by the upper bound of the descending excitation signal that activates motoneuron pools in the spinal cord. Furthermore, a class of movements with scaled velocity profiles can be achieved by tuning the amplitude and duration of this excitation signal. These model predictions agree with a wide body of experimental data obtained from normal human subjects. The results suggest that the control of fast arm movements involves explicit planning for both the equilibrium trajectory and joint stiffness, and that the minimal effort criterion best characterizes the objective of movement planning and control.     

A model of the cerebellar pathways applied to the control of a single-joint robot arm actuated by McKibben artificial muscles

 This article describes an expanded version of a previously proposed motor control scheme, based on rules for combining sensory and motor signals within the central nervous system. Classical control elements of the previous cybernetic circuit were replaced by artificial neural network modules having an architecture based on the connectivity of the cerebellar cortex, and whose functioning is regulated by reinforcement learning. The resulting model was then applied to the motion control of a mechanical, single-joint robot arm actuated by two McKibben artificial muscles. Various biologically plausible learning schemes were studied using both simulations and experiments. After learning, the model was able to accurately pilot the movements of the robot arm, both in velocity and position. Received: 4 September 2000 / Accepted in revised form: 7 November 2001     

Activation of interlimb interactions increases the motor output in legs of healthy subjects: Study under the conditions of arm and leg unloading

The effect of arm movements and movements of individual arm joints on the electrophysiological and kinematic characteristics of voluntary and vibration-triggered stepping-like leg movements was studied under the conditions of horizontal support of the upper and lower limbs. The horizontal support of arms provided a significant increase in the rate of activation of locomotor automatism by noninvasive impact on tonic sensory inputs. The addition of active arm movements during involuntary stepping-like leg movements led to an increase in the EMG activity of hip muscles and was accompanied by an increase in the amplitude of hip and shin movements. The movement of the shoulder joints led to an increase in the activity of hip muscles and was accompanied by an increase in the amplitude of hip and shin movements. Passive arm movements had the same effect on induced leg movements. The movement of the shoulder joints led to an increase in the activity of hip muscles and an increase in the amplitude of movements of knee and hip joints. At the same time, the movement of forearms and wrists had a similar facilitating effect on the physiological and kinematic characteristics of rhythmic stepping-like movements, but influenced the distal segments of legs to a greater extent. Under the conditions of subthreshold vibration of leg muscles, voluntary arm movements led to activation of involuntary rhythmic stepping movements. During voluntary leg movements, the addition of arm movements had a significantly smaller impact on the parameters of rhythmic stepping than during involuntary leg movements. Thus, the simultaneous movements of the upper and lower limbs are an effective method of activation of neural networks connecting the rhythm generators of arms and legs. Under the conditions of arm and leg unloading, the interactions between the cervical and lumbosacral segments of the spinal cord seem to play the major role in the impact of arm movements on the patterns of leg movements. The described methods of activation of interlimb interactions can be used in the rehabilitation of post-stroke patients and patients with spinal cord injuries, Parkinson’s disease, and other neurological diseases.     

Anticipatory postural adjustments to arm movement reveal complex control of paraspinal muscles in the thorax

Anatomical and empirical data suggest that deep and superficial muscles may have different functions for thoracic spine control. This study investigated thoracic paraspinal muscle activity during anticipatory postural adjustments associated with arm movement. Electromyographic (EMG) recordings were made from the right deep (multifidus/rotatores) and superficial (longissimus) muscles at T5, T8, and T11 levels using fine-wire electrodes. Ten healthy participants performed fast unilateral and bilateral flexion and extension arm movements in response to a light. EMG amplitude was measured during 25 ms epochs for 150 ms before and 400 ms after deltoid EMG onset. During arm flexion movements, multifidus and longissimus had two bursts of activity, one burst prior to deltoid and a late burst. With arm extension both muscles were active in a single burst after deltoid onset. There was differential activity with respect to direction of trunk rotation induced by arm movement. Right longissimus was most active with left arm movements and right multifidus was most active with right arm movements. All levels of the thorax responded similarly. We suggest that although thoracic multifidus and longissimus function similarly to control sagittal plane perturbations, these muscles are differentially active with rotational forces on the trunk.     

Estimation of dynamic joint torques and trajectory formation from surface electromyography signals using a neural network model

In this study, human arm movement was re-constructed from electromyography (EMG) signals using a forward dynamics model acquired by an artificial neural network within a modular architecture. Dynamic joint torques at the elbow and shoulder were estimated for movements in the horizontal plane from the surface EMG signals of 10 flexor and extensor muscles. Using only the initial conditions of the arm and the EMG time course as input, the network reliably reconstructed a variety of movement trajectories. The results demonstrate that posture maintenance and multijoint movements, entailing complex via-point specification and co-contraction of muscles, can be accurately computed from multiple surface EMG signals. In addition to the model's empirical uses, such as calculation of arm stiffness during motion, it allows evaluation of hypothesized computational mechanisms of the central nervous system such as virtual trajectory control and optimal trajectory planning.     

Optimality principles for model-based prediction of human gait

Although humans have a large repertoire of potential movements, gait patterns tend to be stereotypical and appear to be selected according to optimality principles such as minimal energy. When applied to dynamic musculoskeletal models such optimality principles might be used to predict how a patient's gait adapts to mechanical interventions such as prosthetic devices or surgery. In this paper we study the effects of different performance criteria on predicted gait patterns using a 2D musculoskeletal model. The associated optimal control problem for a family of different cost functions was solved utilizing the direct collocation method. It was found that fatigue-like cost functions produced realistic gait, with stance phase knee flexion, as opposed to energy-related cost functions which avoided knee flexion during the stance phase. We conclude that fatigue minimization may be one of the primary optimality principles governing human gait.     

A numerical simulation of muscle spindle ensemble encoding during planar movement of the human arm: correlation sensitivity and parameter dependence

We extend the analysis developed in the preceding paper in which we correlated kinematic parameters of planar movements of the human arm made by subjects moving to a visual target with numerical estimates of the ensemble encoding of muscle spindles within some of the muscles of this limb. Three possible models for the inclusion of noise in the calculations of the ensemble encodings are considered: (i) random errors in the angular coordinates from which muscle fascicle, and hence spindle length are calculated, (ii) variability of spindle discharge rates, and (iii) variability in the calculation of the ensemble encoding. In each case the correlations between kinematic variables of the movements and the resultant ensemble encodings decrease as the contribution of the noise term to the calculation of the encodings increases. Subject to the constraint that the magnitude of the noise term remains within physiologically realistic limits, however, the observed correlations persist at statistically significant levels. We also investigate the dependence of the observed correlations on the choice of model parameters, namely (i) the absolute and relative contributions made by simulated spindle primary and secondary afferents to the ensemble encoding, (ii) the inclusion of explicit length-related terms in the model of muscle spindle discharge, and (iii) the fractional power of velocity experienced by the model spindles during movement. The resulting correlations are approximately independent of both the fractional power of velocity and absolute firing levels of both the primary and secondary afferents of the spindle model. The inclusion of explicit length-dependent terms in the model does result in differences in the observed correlation coefficients. In this case, however, the magnitudes of the differences are small. On the basis of these findings we conclude that the correlations between kinematic variables of movement and the associated ensemble encodings are robust with regard to both the choice of model parameters and noise inherent at all stages of the transduction and processing of proprioceptive information. The findings of the present study provide further evidence, therefore, to support the hypothesis that motor structures capable of deriving such an ensemble encoding would be provided with information regarding ongoing movements in both intrinsic (body-centered) and extrinsic (Cartesian) coordinate systems. Received: 17 November 1995 / Accepted in revised form: 17 June 1996     

Impedance characteristics of a neuromusculoskeletal model of the human arm II. Movement control

The modulation of neuromusculoskeletal impedance during movements is analysed using a motor control model of the human arm. The motor control system combines feedback and feedforward control and both control modes are determined in one optimization process. In the model, the stiffness varies at the double movement frequency for 2-Hz oscillatory elbow movements and has high values at the movement reversals. During goal-directed two-degrees-of-freedom arm movements, the stiffness is decreased during the movement and may be increased in the initial and final phases, depending on the movement velocity. The stiffness has a considerable curl during the movement, as was also observed in experimental data. The dynamic stiffness patterns of the model can be explained basically by the α−γ coactivation scheme where feedback gains covary with motor control signals. In addition to the modulation of the gain factors, it is argued that the variation of the intrinsic stiffness has a considerable effect on movement control, especially during fast movements. Received: 14 October 1997 / Accepted in revised form: 18 May 1999