Analysis of an optimal control model of multi-joint arm movements

 In this paper, we propose a model of biological motor control for generation of goal-directed multi-joint arm movements, and study the formation of muscle control inputs and invariant kinematic features of movements. The model has a hierarchical structure that can determine the control inputs for a set of redundant muscles without any inverse computation. Calculation of motor commands is divided into two stages, each of which performs a transformation of motor commands from one coordinate system to another. At the first level, a central controller in the brain accepts instructions from higher centers, which represent the motor goal in the Cartesian space. The controller computes joint equilibrium trajectories and excitation signals according to a minimum effort criterion. At the second level, a neural network in the spinal cord translates the excitation signals and equilibrium trajectories into control commands to three pairs of antagonist muscles which are redundant for a two-joint arm. No inverse computation is required in the determination of individual muscle commands. The minimum effort controller can produce arm movements whose dynamic and kinematic features are similar to those of voluntary arm movements. For fast movements, the hand approaches a target position along a near-straight path with a smooth bell-shaped velocity. The equilibrium trajectories in X and Y show an ‘N’ shape, but the end-point equilibrium path zigzags around the hand path. Joint movements are not always smooth. Joint reversal is found in movements in some directions. The excitation signals have a triphasic (or biphasic) pulse pattern, which leads to stereotyped triphasic (or biphasic) bursts in muscle control inputs, and a dynamically modulated joint stiffness. There is a fixed sequence of muscle activation from proximal muscles to distal muscles. The order is preserved in all movements. For slow movements, it is shown that a constant joint stiffness is necessary to produce a smooth movement with a bell-shaped velocity. Scaled movements can be reproduced by varying the constraints on the maximal level of excitation signals according to the speed of movement. When the inertial parameters of the arm are altered, movement trajectories can be kept invariant by adjusting the pulse height values, showing the ability to adapt to load changes. These results agree with a wide range of experimental observations on human voluntary movements. Received: 4 December 1995 / Accepted in revised form: 17 September 1996     

Velocity-Based Planning of Rapid Elbow Movements Expands the Control Scheme of the Equilibrium Point Hypothesis

According to the equilibrium point hypothesis of voluntary motor control, control action of muscles is not explicitly computed, but rather arises as a consequence of interaction between moving equilibrium position, current kinematics and stiffness of the joint. This approach is attractive as it obviates the need to explicitly specify the forces controlling limb movements. However, many debatable aspects of this hypothesis remain in the manner of specification of the equilibrium point trajectory and muscle activation (or its stiffness), which elicits a restoring force toward the planned equilibrium trajectory. In this study, we expanded the framework of this hypothesis by assuming that the control system uses the velocity measure as the origin of subordinate variables scaling descending commands. The velocity command is translated into muscle control inputs by second order pattern generators, which yield reciprocal command and coactivation commands, and create alternating activation of the antagonistic muscles during movement and coactivation in the post-movement phase, respectively. The velocity command is also integrated to give a position command specifying a moving equilibrium point. This model is purely kinematics-dependent, since the descending commands needed to modulate the visco-elasticity of muscles are implicitly given by simple parametric specifications of the velocity command alone. The simulated movements of fast elbow single-joint movements corresponded well with measured data performed over a wide range of movement distances, in terms of both muscle excitations and kinematics. Our proposal on a synthesis for the equilibrium point approach and velocity command, may offer some insights into the control scheme of the single-joint arm movements.     

On the possibility of linear modelling the human arm neuromuscular apparatus

It has been widely claimed that linear models of the neuromuscular apparatus give very inaccurate approximations of human arm reaching movements. The present paper examines this claim by quantifying the contributions of the various non-linear effects of muscle force generation on the accuracy of linear approximation. We performed computer simulations of a model of a two-joint arm with six monarticular and biarticular muscles. The global actions of individual muscles resulted in a linear dependence of the joint torques on the joint angles and angular velocities, despite the great non-linearity of the muscle properties. The effect of time delay in force generation is much more important for model accuracy than all the non-linear effects, while ignoring this time delay in linear approximation results in large errors. Thus, the viscosity coefficients are rather underestimated and some of them can even be paradoxically estimated to be negative. Similarly, our computation showed that ignoring the time delay resulted in large errors in the estimation of the hand equilibrium trajectory. This could explain why experimentally estimated hand equilibrium trajectories may be complex, even during a simple reaching movement. The hand equilibrium trajectory estimated by a linear model becomes simple when the time delay is taken into account, and it is close to that actually used in the non-linear model. The results therefore provide a theoretical basis for estimating the hand equilibrium trajectory during arm reaching movements and hence for estimating the time course of the motor control signals associated with this trajectory, as set out in the equilibrium point hypothesis. Received: 17 February 1999 / Accepted in revised form: 22 October 1999     

Is a virtual trajectory necessary in reaching movements?

The mass-spring model of limb control was extended to two-joint arm movements in the horizontal plane and tested against kinematic data from human subjects. Two versions of the model were compared in order to test the idea that the equilibrium position of the hand moves along a virtual trajectory as demonstrated in single-joint arm movements (Bizzi et al. 1982, 1984; Latash and Gottlieb 1991). In the peripheral version, the equilibrium position was shifted abruptly, while the torques generated at the joints are gated by rise-time functions. In the central version, the equilibrium position was updated gradually according to a predefined trajectory. The paths and tangential velocity profiles of the hand generated by these two versions were compared to Morasso's (1981) experimental data. The central version generally performed better throughout the workspace except in certain special directions. Moreover, its paths exhibited more stability as the movement speed was varied.     

Kinematic invariants during cyclical arm movements

It has been observed that the motion of the arm end-point (the hand, fingertip or the tip of a pen) is characterized by a number of regularities (kinematic invariants). Trajectory is usually straight, and the velocity profile has a bell shape during point-to-point movements. During drawing movements, a two-thirds power law predicts the dependence of the end-point velocity on the trajectory curvature. Although various principles of movement organization have been discussed as possible origins of these kinematic invariants, the nature of these movement trajectory characteristics remains an open question. A kinematic model of cyclical arm movements derived in the present study analytically demonstrates that all three kinematic invariants can be predicted from a two-joint approximation of the kinematic structure of the arm and from sinusoidal joint motions. With this approach, explicit expressions for two kinematic invariants, the two-thirds power law during drawing movements and the velocity profile during point-to-point movements are obtained as functions of arm segment lengths and joint motion parameters. Additionally, less recognized kinematic invariants are also derived from the model. The obtained analytical expressions are further validated with experimental data. The high accuracy of the predictions confirms practical utility of the model, showing that the model is relevant to human performance over a wide range of movements. The results create a basis for the consolidation of various existing interpretations of kinematic invariants. In particular, optimal control is discussed as a plausible source of invariant characteristics of joint motions and movement trajectories.     

Estimation of dynamic joint torques and trajectory formation from surface electromyography signals using a neural network model

In this study, human arm movement was re-constructed from electromyography (EMG) signals using a forward dynamics model acquired by an artificial neural network within a modular architecture. Dynamic joint torques at the elbow and shoulder were estimated for movements in the horizontal plane from the surface EMG signals of 10 flexor and extensor muscles. Using only the initial conditions of the arm and the EMG time course as input, the network reliably reconstructed a variety of movement trajectories. The results demonstrate that posture maintenance and multijoint movements, entailing complex via-point specification and co-contraction of muscles, can be accurately computed from multiple surface EMG signals. In addition to the model's empirical uses, such as calculation of arm stiffness during motion, it allows evaluation of hypothesized computational mechanisms of the central nervous system such as virtual trajectory control and optimal trajectory planning.     

Controlling the dynamics of a two-joined arm by central patterning and reflex-like processing

It is demonstrated, that a two-joined arm performing goal directed movements is controllable (a) by two central pattern generators (CPG) representing sampled data control, each referring to one joint, initiated for exactly one period, and producing the angular movement width about this joint, and (b) by reflex-like processes operating continuously with respect to time. The latter eliminate the effects of dynamic coupling, gravity, inertia, and mechanical impedance on the movement by proprioceptive feedforward of position, velocity and acceleration signals, thus enabling the CPGs to handle the arm segments as if they were independent and free from forces. Higher ordered centers (with respect to the CPGs) therefore only need to control the kinematics, not dynamics, of the arm.     

Virtual trajectory and stiffness ellipse during multijoint arm movement predicted by neural inverse models

We predict the virtual trajectories and stiffness ellipses during multijoint arm movements by computer simulations. A two-link manipulator with four single-joint muscles and two double-joint muscles is used as a model of the human arm. Physical parameters of the model are derived from several experimental data. Among them, special emphasis is put on low values of the dynamic hand stiffness recently measured during single joint and multijoint movements. The feedback-error-learning scheme to acquire the inverse dynamics model and the inverse statics model is utilized for this prediction. The virtual trajectories are much more complex than the actual trajectories. This indicates that planning the virtual trajectory is as difficult as solving the inverse dynamics problem for medium and fast movements, and simply falsifies the advocated computational advantage of the virtual trajectory control hypothesis. Thus, we conclude that learning inverse models is essential even in the virtual trajectory control framework. Finally, we propose a new computational model to learn the complicated shape of the virtual trajectories by integrating the virtual trajectory control and the feedback-error-learning scheme.     

Walking on a ‘peg leg’: extensor muscle activities and sensory feedback after distal leg denervation in cockroaches

Previous studies in insects demonstrated that leg coordination changes following complete ablation of distal limb segments. However, normal coordination was restored when small peg leg prostheses were attached to leg stumps to permit substrate contact. We have adapted this paradigm to preserve appropriate leg mass and inertia by severing all nerves and muscle tendons in the femur of the cockroach hind leg and converting the animals own limb into a peg leg. Recordings of muscle activities and leg movements before and after denervation showed that: (1) the peg leg is actively used in walking and regular bursts occur in motoneurons to leg extensor muscles; (2) driving of motoneuron activity is sufficient to produce fictive bursting in a muscle whose tendon (apodeme) is cut in the ablation; and (3) similar motoneuron activities are found in walking on an oiled glass surface, when the effects of body weight and mechanical coupling are minimized. When distal segments were completely severed in these preparations, leg use and muscle bursting were disrupted but could be restored if the stumps were pressed against the substrate. These results support the hypothesis that feedback from receptors in proximal leg segments indicating forces allows for active leg use in walking.     

Basic functions of variability of simple pre-planned movements

A model of a pre-planned single joint movements performed without feedback is considered. Modifications of this movement result from transformation of a trajectory pattern f(t) in space and time. The control system adjusts the movement to concrete external conditions specifying values of the transform parameters before the movement performance. The preplanned movement is considered to be simple one, if the transform can be approximated by an affine transform of the movement space and time. In this case, the trajectory of the movement is x(t) = Af(t/ + s) +p, were A and 1/ are space and time scales, s and p are translations. The variability of movements is described by time profiles of variances and covariances of the trajectory x(t), velocity v(t), and acceleration a(t). It is assumed that the variability is defined only by parameters variations. From this assumption follows the main finding of this work: the variability time profiles can be expanded on a special system of basic functions corresponding to established movement parameters. Particularly, basic functions of variance time profiles, reflecting spatial and temporal scaling, are x ²(t) and t ² v ²(t) for trajectory, v ²(t) and (v(t) + t · a(t))² for velocity, and a ²(t) and (2a(t) +t · j(t))², where j(t) = d³ x(t)/dt ³, for acceleration. The variability of a model of a reaching movement was studied analytically. The model predicts certain peculiarities of the form of time profiles (e.g., the variance time profile of velocity is bi-modal, the one of acceleration is tri-modal, etc.). Experimental measurements confirmed predictions. Their consistence allows them to be considered invariant properties of reaching movement. A conclusion can be made, that reaching movement belongs to the type of simple preplanned movements. For a more complex movement, time profiles of variability are also measured and explained by the model of movements of this type. Thus, a movement can be attributed to the type of simple pre-planned ones by testing its variability.     

Postural maintenance during movement: Simulations of a two joint model

Voluntary movements of the upper body are accompanied by anticipatory postural adjustments to the lower body in a standing subject. The long-standing hypothesis is that these anticipatory adjustments serve to counteract the perturbation to the body's center of gravity caused by the voluntary arm movement. This paper presents model simulations investigating the possible roles of anticipatory postural activity that accompanies a rapid, upward arm swing. The model encorporates two (idealized) antagonistic muscle pairs controlling the movements of a double-joint system, with a shoulder joint between the arm and stiff body links, and an ankle joint between the stiff body-leg segment and the ground. Each muscle is represented by a nonlinear viscoelastic element and also includes proprioceptive feedback. Four inputs to the model define the motor control signals for muscle force generation in both the arm and the postural muscle pairs. The neurological component of the model describes consequences of alternate strategies for cocontractions, stretch reflex activity, and anticipatory and synchronous postural activities (or combinations thereof). Simulations with this model show that: (1) none of the postural maintenance schemes considered in these simulations (including varying anticipation) could suppress the initial backward thrust on the body link; (2) the more important destabilizing perturbation is a subsequent forward sway that, left uncountered by postural activity, would eventually leave the body to fall flat on its face; and (3) anticipatory silencing of the postural extensor followed by a brief period of extensor activation (descending control) and synchronous reflex activity (feedback control) appears to be the most likely postural stabilizing strategy that inhibits the continuous forward sway and is consistent with the experimental evidence.     

An equilibrium-point model of electromyographic patterns during single-joint movements based on experimentally reconstructed control signals

The purpose of this work has been to develop a model of electromyographic (EMG) patterns during single-joint movements based on a version of the equilibrium-point hypothesis, a method for experimental reconstruction of the joint compliant characteristics, the dual-strategy hypothesis, and a kinematic model of movement trajectory. EMG patterns are considered emergent properties of hypothetical control patterns that are equally affected by the control signals and peripheral feedback reflecting actual movement trajectory. A computer model generated the EMG patterns based on simulated movement kinematics and hypothetical control signals derived from the reconstructed joint compliant characteristics. The model predictions have been compared to published recordings of movement kinematics and EMG patterns in a variety of movement conditions, including movements over different distances, at different speeds, against different-known inertial loads, and in conditions of possible unexpected decrease in the inertial load. Changes in task parameters within the model led to simulated EMG patterns qualitatively similar to the experimentally recorded EMG patterns. The model's predictive power compares it favourably to the existing models of the EMG patterns.     

Impedance characteristics of a neuromusculoskeletal model of the human arm II. Movement control

The modulation of neuromusculoskeletal impedance during movements is analysed using a motor control model of the human arm. The motor control system combines feedback and feedforward control and both control modes are determined in one optimization process. In the model, the stiffness varies at the double movement frequency for 2-Hz oscillatory elbow movements and has high values at the movement reversals. During goal-directed two-degrees-of-freedom arm movements, the stiffness is decreased during the movement and may be increased in the initial and final phases, depending on the movement velocity. The stiffness has a considerable curl during the movement, as was also observed in experimental data. The dynamic stiffness patterns of the model can be explained basically by the α−γ coactivation scheme where feedback gains covary with motor control signals. In addition to the modulation of the gain factors, it is argued that the variation of the intrinsic stiffness has a considerable effect on movement control, especially during fast movements. Received: 14 October 1997 / Accepted in revised form: 18 May 1999     

Optimal trajectory formation of constrained human arm reaching movements

Opening a door, turning a steering wheel, and rotating a coffee mill are typical examples of human movements that are constrained by the physical environment. The constraints decrease the mobility of the human arm and lead to redundancy in the distribution of actuator forces (either joint torques or muscle forces). Due to this actuator redundancy, there is an infinite number of ways to form a specific arm trajectory. However, humans form trajectories in a unique way. How do humans resolve the redundancy of the constrained motions and specify the hand trajectory? To investigate this problem, we examine human arm movements in a crank-rotation task. To explain the trajectory formation in constrained point-to-point motions, we propose a combined criterion minimizing the hand contact force change and the actuating force change over the course of movement. Our experiments show a close matching between predicted and experimental data.     

Electrical activity of muscles during disturbances in the trajectory of single-joint motion in the arm

Horizontal "extension-flexion" movement of human arm at the elbow joint was studied. Humans performed movements in accordance with the instruction to ignore sudden trajectory changes that arose by the arm during lengthening of different elastic rods with friction. It was shown by multi-dimension regression analysis that the electric activity of biceps and triceps muscles of the shoulder was correlated first of all with the work of load and then with loading force, muscle work and other mechanic and myographic characteristics. One could suppose that information on energy quantities was contained in afferentation signals of the nervous system to correct the program commands on the spinal level. The purpose of correction was to preserve constant quantity of energy expenditure, as well as accuracy and duration of movement.     

Examination of the λ equilibrium point hypothesis when applied to single degree of freedom movements performed with different inertial loads

 One of the theories of human motor control is the λ Equilibrium Point Hypothesis. It is an attractive theory since it offers an easy control scheme where the planned trajectory shifts monotionically from an initial to a final equilibrium state. The feasibility of this model was tested by reconstructing the virtual trajectory and the stiffness profiles for movements performed with different inertial loads and examining them. Three types of movements were tested: passive movements, targeted movements, and repetitive movements. Each of the movements was performed with five different inertial loads. Plausible virtual trajectories and stiffness profiles were reconstructed based on the λ Equilibrium Point Hypothesis for the three different types of movements performed with different inertial loads. However, the simple control strategy supported by the model, where the planned trajectory shifts monotonically from an initial to a final equilibrium state, could not be supported for targeted movements performed with added inertial load. To test the feasibility of the model further we must examine the probability that the human motor control system would choose a trajectory more complicated than the actual trajectory to control. Received: 20 June 1995 / Accepted in revised form: 6 August 1996     

Reproduction of tracking movements and target positioning of the forearm in humans in the absence of visual control

In 17 healthy volunteers, we studied movements of the forearm, which included episodes of positioning on the target level. The trajectory of the non-ballistic (relatively slow) movement looked like a double trapezium (flexion of the elbow joint from the state of full extension, 0 deg, positioning on the 50 deg level, further flexion to the limit angle of 100 deg, and a similar reverse sequence). The command trajectory and the trajectory of the realized movement were visualized with movements of cursors on a monitor in time/joint angle coordinates. We compared parameters of the tracking movements (in the presence of visual feedback) and their blindfold reproduction (with the complete absence of visual control). It was found that blindfold reproduction movements differ from sample tracking movements and their reproduction with partial limitation of visual control [16] in higher peak velocities and shorter durations, i.e., a trend toward conversion of such movements into ballistic ones was observed. Under conditions of elimination of visual control, movements that led to positioning were mostly hypermetric, i.e., positioning was usually accompanied by positive systematic errors (whose sign coincided with the direction of the preceding movement phase). The mean intragroup value of the systematic error of the first positioning (after flexion to the target level) was +6.73 ± 1.15 deg, while the respective mean for the second positioning (after extension to the same level) was +4.00 ± 1.31 deg. The nonlinear properties of stretch receptors of muscles whose activity provides the formation of a proprioceptive estimate of the joint angle are considered the crucial reason for systematic errors of blindfold positioning.Neirofiziologiya/Neurophysiology, Vol. 36, Nos. 5/6, pp. 393–404, September–December, 2004.This revised version was published online in April 2005 with a corrected cover date and copyright year.     

Moving just like you: motor interference depends on similar motility of agent and observer

Recent findings in neuroscience suggest an overlap between brain regions involved in the execution of movement and perception of another's movement. This so-called "action-perception coupling" is supposed to serve our ability to automatically infer the goals and intentions of others by internal simulation of their actions. A consequence of this coupling is motor interference (MI), the effect of movement observation on the trajectory of one's own movement. Previous studies emphasized that various features of the observed agent determine the degree of MI, but could not clarify how human-like an agent has to be for its movements to elicit MI and, more importantly, what 'human-like' means in the context of MI. Thus, we investigated in several experiments how different aspects of appearance and motility of the observed agent influence motor interference (MI). Participants performed arm movements in horizontal and vertical directions while observing videos of a human, a humanoid robot, or an industrial robot arm with either artificial (industrial) or human-like joint configurations. Our results show that, given a human-like joint configuration, MI was elicited by observing arm movements of both humanoid and industrial robots. However, if the joint configuration of the robot did not resemble that of the human arm, MI could longer be demonstrated. Our findings present evidence for the importance of human-like joint configuration rather than other human-like features for perception-action coupling when observing inanimate agents.     

Interaction torque contributes to planar reaching at slow speed

Background

How the central nervous system (CNS) organizes the joint dynamics for multi-joint movement is a complex problem, because of the passive interaction among segmental movements. Previous studies have demonstrated that the CNS predictively compensates for interaction torque (INT) which is arising from the movement of the adjacent joints. However, most of these studies have mainly examined quick movements, presumably because the current belief is that the effects of INT are not significant at slow speeds. The functional contribution of INT for multijoint movements performed in various speeds is still unclear. The purpose of this study was to examine the contribution of INT to a planer reaching in a wide range of motion speeds for healthy subjects.

Methods

Subjects performed reaching movements toward five targets under three different speed conditions. Joint position data were recorded using a 3-D motion analysis device (50 Hz). Torque components, muscle torque (MUS), interaction torque (INT), gravity torque (G), and net torque (NET) were calculated by solving the dynamic equations for the shoulder and elbow. NET at a joint which produces the joint kinematics will be an algebraic sum of torque components; NET = MUS - G - INT. Dynamic muscle torque (DMUS = MUS-G) was also calculated. Contributions of INT impulse and DMUS impulse to NET impulse were examined.

Results

The relative contribution of INT to NET was not dependent on speed for both joints at every target. INT was additive (same direction) to DMUS at the shoulder joint, while in the elbow DMUS counteracted (opposed to) INT. The trajectory of reach was linear and two-joint movements were coordinated with a specific combination at each target, regardless of motion speed. However, DMUS at the elbow was opposed to the direction of elbow movement, and its magnitude varied from trial to trial in order to compensate for the variability of INT.

Conclusion

Interaction torque was important at slow speeds. Muscle torques at the two joints were not directly related to each other to produce coordinated joint movement during a reach. These results support Bernstein's idea that coordinated movement is not completely determined by motor command in multi-joint motion. Based on the data presented in this study and the work of others, a model for the connection between joint torques (muscle and passive torques including interaction torque) and joint coordination is proposed.