A model of the cerebellar pathways applied to the control of a single-joint robot arm actuated by McKibben artificial muscles

 This article describes an expanded version of a previously proposed motor control scheme, based on rules for combining sensory and motor signals within the central nervous system. Classical control elements of the previous cybernetic circuit were replaced by artificial neural network modules having an architecture based on the connectivity of the cerebellar cortex, and whose functioning is regulated by reinforcement learning. The resulting model was then applied to the motion control of a mechanical, single-joint robot arm actuated by two McKibben artificial muscles. Various biologically plausible learning schemes were studied using both simulations and experiments. After learning, the model was able to accurately pilot the movements of the robot arm, both in velocity and position. Received: 4 September 2000 / Accepted in revised form: 7 November 2001     

Analysis of an optimal control model of multi-joint arm movements

 In this paper, we propose a model of biological motor control for generation of goal-directed multi-joint arm movements, and study the formation of muscle control inputs and invariant kinematic features of movements. The model has a hierarchical structure that can determine the control inputs for a set of redundant muscles without any inverse computation. Calculation of motor commands is divided into two stages, each of which performs a transformation of motor commands from one coordinate system to another. At the first level, a central controller in the brain accepts instructions from higher centers, which represent the motor goal in the Cartesian space. The controller computes joint equilibrium trajectories and excitation signals according to a minimum effort criterion. At the second level, a neural network in the spinal cord translates the excitation signals and equilibrium trajectories into control commands to three pairs of antagonist muscles which are redundant for a two-joint arm. No inverse computation is required in the determination of individual muscle commands. The minimum effort controller can produce arm movements whose dynamic and kinematic features are similar to those of voluntary arm movements. For fast movements, the hand approaches a target position along a near-straight path with a smooth bell-shaped velocity. The equilibrium trajectories in X and Y show an ‘N’ shape, but the end-point equilibrium path zigzags around the hand path. Joint movements are not always smooth. Joint reversal is found in movements in some directions. The excitation signals have a triphasic (or biphasic) pulse pattern, which leads to stereotyped triphasic (or biphasic) bursts in muscle control inputs, and a dynamically modulated joint stiffness. There is a fixed sequence of muscle activation from proximal muscles to distal muscles. The order is preserved in all movements. For slow movements, it is shown that a constant joint stiffness is necessary to produce a smooth movement with a bell-shaped velocity. Scaled movements can be reproduced by varying the constraints on the maximal level of excitation signals according to the speed of movement. When the inertial parameters of the arm are altered, movement trajectories can be kept invariant by adjusting the pulse height values, showing the ability to adapt to load changes. These results agree with a wide range of experimental observations on human voluntary movements. Received: 4 December 1995 / Accepted in revised form: 17 September 1996     

Beyond Muscles Stiffness: Importance of State-Estimation to Account for Very Fast Motor Corrections

Feedback delays are a major challenge for any controlled process, and yet we are able to easily control limb movements with speed and grace. A popular hypothesis suggests that the brain largely mitigates the impact of feedback delays (∼50 ms) by regulating the limb intrinsic visco-elastic properties (or impedance) with muscle co-contraction, which generates forces proportional to changes in joint angle and velocity with zero delay. Although attractive, this hypothesis is often based on estimates of limb impedance that include neural feedback, and therefore describe the entire motor system. In addition, this approach does not systematically take into account that muscles exhibit high intrinsic impedance only for small perturbations (short-range impedance). As a consequence, it remains unclear how the nervous system handles large perturbations, as well as disturbances encountered during movement when short-range impedance cannot contribute. We address this issue by comparing feedback responses to load pulses applied to the elbow of human subjects with theoretical simulations. After validating the model parameters, we show that the ability of humans to generate fast and accurate corrective movements is compatible with a control strategy based on state estimation. We also highlight the merits of delay-uncompensated robust control, which can mitigate the impact of internal model errors, but at the cost of slowing feedback corrections. We speculate that the puzzling observation of presynaptic inhibition of peripheral afferents in the spinal cord at movement onset helps to counter the destabilizing transition from high muscle impedance during posture to low muscle impedance during movement.     

A neural network model of the cerebellar cortex performing dynamic associations

The present paper proposes a model which applies formal neural network modeling techniques to construct a theoretical representation of the cerebellar cortex and its performances in motor control. A schema that makes explicit use of propagation delays of neural signals, is introduced to describe the ability to store temporal sequences of patterns in the Golgi-granule cell system. A perceptron association is then performed on these sequences of patterns by the Purkinje cell layer. The model conforms with important biological constraints, such as the known excitatory or inhibitory nature of the various synapses. Also, as suggested by experimental evidence, the synaptic plasticity underlying the learning ability of the model, is confined to the parallel fiber — Purkinje cell synapses, and takes place under the control of the climbing fibers. The result is a neural network model, constructed according to the anatomy of the cerebellar cortex, and capable of learning and retrieval of temporal sequences of patterns. It provides a framework to represent and interpret properties of learning and control of movements by the cerebellum, and to assess the capacity of formal neural network techniques for modeling of real neural systems.     

Impedance characteristics of a neuromusculoskeletal model of the human arm II. Movement control

The modulation of neuromusculoskeletal impedance during movements is analysed using a motor control model of the human arm. The motor control system combines feedback and feedforward control and both control modes are determined in one optimization process. In the model, the stiffness varies at the double movement frequency for 2-Hz oscillatory elbow movements and has high values at the movement reversals. During goal-directed two-degrees-of-freedom arm movements, the stiffness is decreased during the movement and may be increased in the initial and final phases, depending on the movement velocity. The stiffness has a considerable curl during the movement, as was also observed in experimental data. The dynamic stiffness patterns of the model can be explained basically by the α−γ coactivation scheme where feedback gains covary with motor control signals. In addition to the modulation of the gain factors, it is argued that the variation of the intrinsic stiffness has a considerable effect on movement control, especially during fast movements. Received: 14 October 1997 / Accepted in revised form: 18 May 1999     

Neural computations underlying the exertion of force: a model

We have developed a model that simulates possible mechanisms by which supraspinal neuronal signals coding forces could converge in the spinal cord and provide an ongoing integrated signal to the motoneuronal pools whose activation results in the exertion of force. The model consists of a three-layered neural network connected to a two-joint-six-muscle model of the arm. The network layers represent supraspinal populations, spinal cord interneurons, and motoneuronal pools. We propose an approach to train the network so that, after the synaptic connections between the layers are adjusted, the performance of the model is consistent with experimental data obtained on different organisms using different experimental paradigms: the stiffness characteristics of human arm; the structure of force fields generated by the stimulation of the frog's spinal cord; and a correlation between motor cortical activity and force exerted by monkey against an immovable object. The model predicts a specific pattern of connections between supraspinal populations coding forces and spinal cord interneurons: the weight of connection should be correlated with directional preference of interconnected units. Finally, our simulations demonstrate that the force generated by the sum of neural signals can be nearly equal to the vector sum of forces generated by each signal independently, in spite of the complex nonlinearities intervening between supraspinal commands and forces exerted by the arm in response to these commands.     

Simultaneous Brain–Cervical Cord fMRI Reveals Intrinsic Spinal Cord Plasticity during Motor Sequence Learning

The spinal cord participates in the execution of skilled movements by translating high-level cerebral motor representations into musculotopic commands. Yet, the extent to which motor skill acquisition relies on intrinsic spinal cord processes remains unknown. To date, attempts to address this question were limited by difficulties in separating spinal local effects from supraspinal influences through traditional electrophysiological and neuroimaging methods. Here, for the first time, we provide evidence for local learning-induced plasticity in intact human spinal cord through simultaneous functional magnetic resonance imaging of the brain and spinal cord during motor sequence learning. Specifically, we show learning-related modulation of activity in the C6–C8 spinal region, which is independent from that of related supraspinal sensorimotor structures. Moreover, a brain–spinal cord functional connectivity analysis demonstrates that the initial linear relationship between the spinal cord and sensorimotor cortex gradually fades away over the course of motor sequence learning, while the connectivity between spinal activity and cerebellum gains strength. These data suggest that the spinal cord not only constitutes an active functional component of the human motor learning network but also contributes distinctively from the brain to the learning process. The present findings open new avenues for rehabilitation of patients with spinal cord injuries, as they demonstrate that this part of the central nervous system is much more plastic than assumed before. Yet, the neurophysiological mechanisms underlying this intrinsic functional plasticity in the spinal cord warrant further investigations.     

Learning to predict the future: the cerebellum adapts feedforward movement control

The role of the cerebellum in motor control and learning has been largely inferred from the effects of cerebellar damage. Recent work shows that cerebellar damage produces greater impairment of movements that require predictive as opposed to reactive control. This dissociation is consistent across many different types of movement. Predictive control is crucial for fast and ballistic movements, but impaired prediction can also affect slow movements, because of increased reliance on time-delayed feedback signals. The new findings are compatible with theories of cerebellar function, but still do not resolve whether the cerebellum operates by predicting the optimal motor commands or future sensory states. Prediction mechanisms must be learned and maintained through comparisons between predicted and observed outcomes. New results show that not all such error information is equivalent in driving cerebellar learning.     

A model of activity-dependent formation of cerebellar microzones

According to modern views of the cerebellum in motor control, each cerebellar functional unit, or microzone, learns how to execute predictive and coordinative control, based on long-term depression of the granule cell-Purkinje cell synapses. In the present paper, in light of recent experimental and theoretical studies on synaptic elimination and cerebellar motor learning, a model of the formation of cerebellar microzones by climbing fiber synaptic elimination is proposed. It is shown that competition for an activity-dependent supply of neurotrophic factor can reproduce the spatio-temporal characteristics of climbing fiber synaptic elimination. It is further shown that when this elimination is accurate, motor coordination can be acquired in an arm reaching task. In view of the results of the present study, several predictions are proposed. Received: 19 January 1998 / Accepted in revised form: 22 April 1998     

Modelling of direct motor program learning in fast human arm motions

We propose and simulate a new paradigm for organization of motor control in fast and accurate human arm motions. We call the paradigm direct motor program learning since the control programs are learned directly without knowing or learning the dynamics of a controlled system.The idea is to approximate the dependence of the motor control programs on the vector of the task parameters rather than to use a model of the system dynamics. We apply iterative learning control and scattered data multivariate approximation techniques to achieve the goal. The advantage of the paradigm is that the control complexity depends neither on the order nor on the nonlinearity of the system dynamics.We simulate the direct motor program learning paradigm in the task of point-to-point control of fast planar human arm motions. Simulation takes into account nonlinear arm dynamics, muscle force dynamics, delay in low-level reflex feedback, time dependence of the feedback gains and coactivation of antagonist muscles. Despite highly nonlinear time-variant dynamics of the controlled system, reasonably good motion precision is obtained over a wide range of the task parameters (initial and final positions of the arm). The simulation results demonstrate that the paradigm is indeed viable and could be considered as a possible explanation for the organization of motor control of fast motions.     

Impedance characteristics of a neuromusculoskeletal model of the human arm I. Posture control

The mechanical impedance of neuromusculoskeletal models of the human arm is studied in this paper. The model analysis provides a better understanding of the contributions of possible intrinsic and reflexive components of arm impedance, makes clear the limitations of second-order mass-viscosity-stiffness models and reveals possible task effects on the impedance. The musculoskeletal model describes planar movements of the upper arm and forearm, which are moved by six lumped muscles with nonlinear dynamics. The motor control system is represented by a neural network which combines feedforward and feedback control. It is optimized for the control of movements or for posture control in the presence of external forces. The achieved impedance characteristics depend on the conditions during the learning process. In particular, the impedance is adapted in a suitable way to the frequency content and direction of external forces acting on the hand during an isometric task. The impedance characteristics of a model, which is optimized for movement control, are similar to experimental data in the literature. The achieved stiffness is, to a large extent, reflexively determined whereas the approximated viscosity is primarily due to intrinsic attributes. It is argued that usually applied Hill-type muscle models do not properly represent intrinsic muscle stiffness. Received: 14 October 1997 / Accepted in revised form: 18 May 1999     

Statistical characteristics of climbing fiber spikes necessary for efficient cerebellar learning

 Mean firing rates (MFRs), with analogue values, have thus far been used as information carriers of neurons in most brain theories of learning. However, the neurons transmit the signal by spikes, which are discrete events. The climbing fibers (CFs), which are known to be essential for cerebellar motor learning, fire at the ultra-low firing rates (around 1 Hz), and it is not yet understood theoretically how high-frequency information can be conveyed and how learning of smooth and fast movements can be achieved. Here we address whether cerebellar learning can be achieved by CF spikes instead of conventional MFR in an eye movement task, such as the ocular following response (OFR), and an arm movement task. There are two major afferents into cerebellar Purkinje cells: parallel fiber (PF) and CF, and the synaptic weights between PFs and Purkinje cells have been shown to be modulated by the stimulation of both types of fiber. The modulation of the synaptic weights is regulated by the cerebellar synaptic plasticity. In this study we simulated cerebellar learning using CF signals as spikes instead of conventional MFR. To generate the spikes we used the following four spike generation models: (1) a Poisson model in which the spike interval probability follows a Poisson distribution, (2) a gamma model in which the spike interval probability follows the gamma distribution, (3) a max model in which a spike is generated when a synaptic input reaches maximum, and (4) a threshold model in which a spike is generated when the input crosses a certain small threshold. We found that, in an OFR task with a constant visual velocity, learning was successful with stochastic models, such as Poisson and gamma models, but not in the deterministic models, such as max and threshold models. In an OFR with a stepwise velocity change and an arm movement task, learning could be achieved only in the Poisson model. In addition, for efficient cerebellar learning, the distribution of CF spike-occurrence time after stimulus onset must capture at least the first, second and third moments of the temporal distribution of error signals. Received: 28 January 2000 / Accepted in revised form: 2 August 2000     

Computational nature of human adaptive control during learning of reaching movements in force fields

Learning to make reaching movements in force fields was used as a paradigm to explore the system architecture of the biological adaptive controller. We compared the performance of a number of candidate control systems that acted on a model of the neuromuscular system of the human arm and asked how well the dynamics of the candidate system compared with the movement characteristics of 16 subjects. We found that control via a supra-spinal system that utilized an adaptive inverse model resulted in dynamics that were similar to that observed in our subjects, but lacked essential characteristics. These characteristics pointed to a different architecture where descending commands were influenced by an adaptive forward model. However, we found that control via a forward model alone also resulted in dynamics that did not match the behavior of the human arm. We considered a third control architecture where a forward model was used in conjunction with an inverse model and found that the resulting dynamics were remarkably similar to that observed in the experimental data. The essential property of this control architecture was that it predicted a complex pattern of near-discontinuities in hand trajectory in the novel force field. A nearly identical pattern was observed in our subjects, suggesting that generation of descending motor commands was likely through a control system architecture that included both adaptive forward and inverse models. We found that as subjects learned to make reaching movements, adaptation rates for the forward and inverse models could be independently estimated and the resulting changes in performance of subjects from movement to movement could be accurately accounted for. Results suggested that the adaptation of the forward model played a dominant role in the motor learning of subjects. After a period of consolidation, the rates of adaptation in the internal models were significantly larger than those observed before the memory had consolidated. This suggested that consolidation of motor memory coincided with freeing of certain computational resources for subsequent learning. Received: 01 January 1998 / Accepted in revised form: 26 January 1999     

Complex predictive eye pursuit in monkey: a model system for cerebellar studies of skilled movement

Smooth pursuit eye movements provide a good model system for cerebellar studies of complex motor control in monkeys. First, the pursuit system exhibits predictive control along complex trajectories and this control improves with training. Second, the flocculus/paraflocculus region of the cerebellum appears to generate this control. Lesions impair pursuit and neural activity patterns are closely related to eye motion during complex pursuit. Importantly, neural responses lead eye motion during predictive pursuit and lag eye motion during non-predictable target motions that require visual control. The idea that flocculus/paraflocculus predictive control is non-visual is also supported by a lack of correlation between neural activity and retinal image motion during pursuit. Third, biologically accurate neural network models of the flocculus/paraflocculus allow the exploration and testing of pursuit mechanisms. Our current model can generate predictive control without visual input in a manner that is compatible with the extensive experimental data available for this cerebellar system. Similar types of non-visual cerebellar control are likely to facilitate the wide range of other skilled movements that are observed.     

Influence of Vibration on Motor Responses in the Upper Arm Muscles under Stationary Conditions and during Voluntary and Evoked Arm Movements under Limb Unloading Conditions

Locomotion of mammals, including humans, is based on the rhythmic activity of spinal cord circuitries. The functioning of these circuitries depends on multimodal afferent information and on supraspinal influences from the motor cortex. Using the method of transcranial magnetic stimulation (TMS) of arm muscle areas in the motor cortex, we studied the motor evoked potentials (MEP) in the upper arm muscles in stationary conditions and during voluntary and vibration-evoked arm movements. The study included 13 healthy subjects under arm and leg unloading conditions. In the first series of experiments, with motionless limbs, the effect of vibration of left upper arm muscles on motor responses in these muscles was evaluated. In the second series of experiments, MEP were compared in the same muscles during voluntary and rhythmic movements generated by left arm m. triceps brachii vibration (the right arm was stationary). Motionless left arm vibration led to an increase in MEP values in both vibrated muscle and in most of the non-vibrated muscles. For most target muscles, MEP was greater with voluntary arm movements than with vibration-evoked movements. At the same time, a similar MEP modulation in the cycle of arm movements was observed in the same upper arm muscles during both types of arm movements. TMS of the motor cortex significantly potentiated arm movements generated by vibration, but its effect on voluntary movements was weaker. These results indicate significant differences in the degree of motor cortex involvement in voluntary and evoked arm movements. We suppose that evoked arm movements are largely due to spinal rather than central mechanisms of generation of rhythmic movements.     

Patterns of motor activity in the isolated nerve cord of the octopus arm

The extremely flexible octopus arm provides a unique opportunity for studying movement control in a highly redundant motor system. We describe a novel preparation that allows analysis of the peripheral nervous system of the octopus arm and its interaction with the muscular and mechanosensory elements of the arm's intrinsic muscular system. First we examined the synaptic responses in muscle fibers to identify the motor pathways from the axial nerve cord of the arm to the surrounding musculature. We show that the motor axons project to the muscles via nerve roots originating laterally from the arm nerve cord. The motor field of each nerve is limited to the region where the nerve enters the arm musculature. The same roots also carry afferent mechanosensory information from the intrinsic muscle to the axial nerve cord. Next, we characterized the pattern of activity generated in the dorsal roots by electrically stimulating the axial nerve cord. The evoked activity, although far reaching and long lasting, cannot alone account for the arm extension movements generated by similar electrical stimulation. The mismatch between patterns of activity in the isolated cord and in an intact arm may stem from the involvement of mechanosensory feedback in natural arm extension.     

Sensorimotor regulation of movements: Novel strategies for the recovery of mobility

A series of observations have provided important insight into properties of the spinal as well as supraspinal circuitries that control posture and movement. We have demonstrated that spinal rats can regain full weight-bearing standing and stepping over a range of speeds and directions with the aid of electrically enabling motor control (eEmc), pharmacological modulation (fEmc), and training [1, 2]. Also, we have reported that voluntary control movements of individual joints and limbs can be regained after complete paralysis in humans [3, 4]. However, the ability to generate significant levels of voluntary weight-bearing stepping with or without epidural spinal cord stimulation remains limited. Herein we introduce a novel method of painless transcutaneous electrical enabling motor control (pcEmc) and sensory enabling motor control (sEmc) strategy to neuromodulate the physiological state of the spinal cord. We have found that a combination of a novel non-invasive transcutaneous spinal cord stimulation and sensory-motor stimulation of leg mechanoreceptors can modulate the spinal locomotor circuitry to state that enables voluntary rhythmic locomotor movements.     

A neural network model for the intersensory coordination involved in goal-directed movements

A neural network model for a sensorimotor system, which was developed to simulate oriented movements in man, is presented. It is composed of a formal neural network comprising two layers: a sensory layer receiving and processing sensory inputs, and a motor layer driving a simulated arm. The sensory layer is an extension of the topological network previously proposed by Kohonen (1984). Two kinds of sensory modality, proprioceptive and exteroceptive, are used to define the arm position. Each sensory cell receives proprioceptive inputs provided by each arm-joint together with the exteroceptive inputs. This sensory layer is therefore a kind of associative layer which integrates two separate sensory signals relating to movement coding. It is connected to the motor layer by means of adaptive synapses which provide a physical link between a motor activity and its sensory consequences. After a learning period, the spatial map which emerges in the sensory layer clearly depends on the sensory inputs and an associative map of both the arm and the extra-personal space is built up if proprioceptive and exteroceptive signals are processed together. The sensorimotor transformations occuring in the junctions linking the sensory and motor layers are organized in such a manner that the simulated arm becomes able to reach towards and track a target in extra-personal space. Proprioception serves to determine the final arm posture adopted and to correct the ongoing movement in cases where changes in the target location occur. With a view to developing a sensorimotor control system with more realistic salient features, a robotic model was coupled with the formal neural network. This robotic implementation of our model shows the capacity of formal neural networks to control the displacement of mechanical devices.     

Intermediate representations in the formation of arm trajectories

Psychophysical evidence shows that the planning of an arm trajectory is specified by the central nervous system in extrinsic coordinates. The complex issue of translating the planning of arm movements into muscle forces is discussed in relation to the recent discovery of structures in the brainstem and in the spinal cord. These structures represent discrete maps of motor behavior. Remarkably, the force outputs, produced by activating different zones of the map, sumvectorially. This vectorial combination of motor outputs is a mechanism for producing a vast repertoire of motor behaviors in a simple fashion.