Analysis of an optimal control model of multi-joint arm movements

 In this paper, we propose a model of biological motor control for generation of goal-directed multi-joint arm movements, and study the formation of muscle control inputs and invariant kinematic features of movements. The model has a hierarchical structure that can determine the control inputs for a set of redundant muscles without any inverse computation. Calculation of motor commands is divided into two stages, each of which performs a transformation of motor commands from one coordinate system to another. At the first level, a central controller in the brain accepts instructions from higher centers, which represent the motor goal in the Cartesian space. The controller computes joint equilibrium trajectories and excitation signals according to a minimum effort criterion. At the second level, a neural network in the spinal cord translates the excitation signals and equilibrium trajectories into control commands to three pairs of antagonist muscles which are redundant for a two-joint arm. No inverse computation is required in the determination of individual muscle commands. The minimum effort controller can produce arm movements whose dynamic and kinematic features are similar to those of voluntary arm movements. For fast movements, the hand approaches a target position along a near-straight path with a smooth bell-shaped velocity. The equilibrium trajectories in X and Y show an ‘N’ shape, but the end-point equilibrium path zigzags around the hand path. Joint movements are not always smooth. Joint reversal is found in movements in some directions. The excitation signals have a triphasic (or biphasic) pulse pattern, which leads to stereotyped triphasic (or biphasic) bursts in muscle control inputs, and a dynamically modulated joint stiffness. There is a fixed sequence of muscle activation from proximal muscles to distal muscles. The order is preserved in all movements. For slow movements, it is shown that a constant joint stiffness is necessary to produce a smooth movement with a bell-shaped velocity. Scaled movements can be reproduced by varying the constraints on the maximal level of excitation signals according to the speed of movement. When the inertial parameters of the arm are altered, movement trajectories can be kept invariant by adjusting the pulse height values, showing the ability to adapt to load changes. These results agree with a wide range of experimental observations on human voluntary movements. Received: 4 December 1995 / Accepted in revised form: 17 September 1996     

Maximization of learning speed in the motor cortex due to neuronal redundancy

Many redundancies play functional roles in motor control and motor learning. For example, kinematic and muscle redundancies contribute to stabilizing posture and impedance control, respectively. Another redundancy is the number of neurons themselves; there are overwhelmingly more neurons than muscles, and many combinations of neural activation can generate identical muscle activity. The functional roles of this neuronal redundancy remains unknown. Analysis of a redundant neural network model makes it possible to investigate these functional roles while varying the number of model neurons and holding constant the number of output units. Our analysis reveals that learning speed reaches its maximum value if and only if the model includes sufficient neuronal redundancy. This analytical result does not depend on whether the distribution of the preferred direction is uniform or a skewed bimodal, both of which have been reported in neurophysiological studies. Neuronal redundancy maximizes learning speed, even if the neural network model includes recurrent connections, a nonlinear activation function, or nonlinear muscle units. Furthermore, our results do not rely on the shape of the generalization function. The results of this study suggest that one of the functional roles of neuronal redundancy is to maximize learning speed.     

Modular neuromuscular control of human locomotion by central pattern generator

The central pattern generators (CPG) in the spinal cord are thought to be responsible for producing the rhythmic motor patterns during rhythmic activities. For locomotor tasks, this involves much complexity, due to a redundant system of muscle actuators with a large number of highly nonlinear muscles. This study proposes a reduced neural control strategy for the CPG, based on modular organization of the co-active muscles, i.e., muscle synergies. Four synergies were extracted from the EMG data of the major leg muscles of two subjects, during two gait trials each, using non-negative matrix factorization algorithm. A Matsuoka׳s four-neuron CPG model with mutual inhibition, was utilized to generate the rhythmic activation patterns of the muscle synergies, using the hip flexion angle and foot contact force information from the sensory afferents as inputs. The model parameters were tuned using the experimental data of one gait trial, which resulted in a good fitting accuracy (RMSEs between 0.0491 and 0.1399) between the simulation and experimental synergy activations. The model׳s performance was then assessed by comparing its predictions for the activation patterns of the individual leg muscles during locomotion with the relevant EMG data. Results indicated that the characteristic features of the complex activation patterns of the muscles were well reproduced by the model for different gait trials and subjects. In general, the CPG- and muscle synergy-based model was promising in view of its simple architecture, yet extensive potentials for neuromuscular control, e.g., resolving redundancies, distributed and fast control, and modulation of locomotion by simple control signals.     

A neural network model for the intersensory coordination involved in goal-directed movements

A neural network model for a sensorimotor system, which was developed to simulate oriented movements in man, is presented. It is composed of a formal neural network comprising two layers: a sensory layer receiving and processing sensory inputs, and a motor layer driving a simulated arm. The sensory layer is an extension of the topological network previously proposed by Kohonen (1984). Two kinds of sensory modality, proprioceptive and exteroceptive, are used to define the arm position. Each sensory cell receives proprioceptive inputs provided by each arm-joint together with the exteroceptive inputs. This sensory layer is therefore a kind of associative layer which integrates two separate sensory signals relating to movement coding. It is connected to the motor layer by means of adaptive synapses which provide a physical link between a motor activity and its sensory consequences. After a learning period, the spatial map which emerges in the sensory layer clearly depends on the sensory inputs and an associative map of both the arm and the extra-personal space is built up if proprioceptive and exteroceptive signals are processed together. The sensorimotor transformations occuring in the junctions linking the sensory and motor layers are organized in such a manner that the simulated arm becomes able to reach towards and track a target in extra-personal space. Proprioception serves to determine the final arm posture adopted and to correct the ongoing movement in cases where changes in the target location occur. With a view to developing a sensorimotor control system with more realistic salient features, a robotic model was coupled with the formal neural network. This robotic implementation of our model shows the capacity of formal neural networks to control the displacement of mechanical devices.     

Distributed parallel processing in the vertical vestibulo-ocular reflex: Learning networks compared to tensor theory

The vestibulo-ocular reflex (VOR) is capable of producing compensatory eye movements in three dimensions. It utilizes the head rotational velocity signals from the semicircular canals to control the contractions of the extraocular muscles. Since canal and muscle coordinate frames are not orthogonal and differ from one another, a sensorimotor transformation must be produced by the VOR neural network. Tensor theory has been used to construct a linear transformation that can model the three-dimensional behavior of the VOR. But tensor theory does not take the distributed, redundant nature of the VOR neural network into account. It suggests that the neurons subserving the VOR, such as vestibular nucleus neurons, should have specific sensitivity-vectors. Actual data, however, are not in accord. Data from the cat show that the sensitivity-vectors of vestibular nucleus neurons, rather than aligning with any specific vectors, are dispersed widely. As an alternative to tensor theory, we modeled the vertical VOR as a three-layered neural network programmed using the back-propagation learning algorithm. Units in mature networks had divergent sensitivity-vectors which resembled those of actual vestibular nucleus neurons in the cat. This similarity suggests that the VOR sensorimotor transformation may be represented redundantly rather than uniquely. The results demonstrate how vestibular nucleus neurons can encode the VOR sensorimotor transformation in a distributed manner.     

A model for neural control of gradation of muscle force

A mathematical muscle model is presented that relates neural control signals linearly to muscle force without violating important known physiological constraints, such as the size-principle (Henneman and Mendell 1981) and non-linear twitch summation (Burke et al. 1976). This linearity implies that the neural control signals (defined as a weighted sum of activities in a nerve bundle) can be interpreted as the internal representation of total muscle force. The model allows for different relative contributions from the two force-grading mechanisms, i.e. the recruitment of motor units and the modulation of their firing frequency. It can therefore be applied to a variety of (distal and proximal) muscles. Furthermore, it permits simple mechanisms for controlling muscle force, e.g. in superposed motor tasks. The model confirms our intuitive notion that a weighted sum of activities in a nerve bundle can directly represent an external controlled variable, which in this case is exerted muscle force.     

Influence of nerve supply on hand electromyography coherence during a three-digit task

Intermuscular coupling has been investigated to understand neural inputs to coordinate muscles in a motor performance. However, little is known on the role of nerve innervation on intermuscular coupling. The purpose of this study was to investigate how the anatomy of nerve distribution affected intermuscular coupling in the hand during static grip. Electromyographic (EMG) signals were recorded from intrinsic and extrinsic muscles while subjects performed a static grip. Coherence was computed for muscle pairs innervated by either the same or different nerves. The results did not support the hypothesis that muscles sharing the same nerve exhibit greater coupling than muscles innervated by different nerves. In general, extrinsic muscle pairs displayed higher coherence than intrinsic pairs. The results suggest that intermuscular coupling in a voluntary motor task is likely modulated in a functional manner and that different nerves might transport common neural inputs to functionally coupled muscles.     

A model for learning human reaching movements

 Reaching movement is a fast movement towards a given target. The main characteristics of such a movement are straight path and a bell-shaped speed profile. In this work a mathematical model for the control of the human arm during ballistic reaching movements is presented. The model of the arm contains a 2 degrees of freedom planar manipulator, and a Hill-type, non-linear mechanical model of six muscles. The arm model is taken from the literature with minor changes. The nervous system is modeled as an adjustable pattern generator that creates the control signals to the muscles. The control signals in this model are rectangular pulses activated at various amplitudes and timings, that are determined according to the given target. These amplitudes and timings are the parameters that should be related to each target and initial conditions in the workspace. The model of the nervous system consists of an artificial neural net that maps any given target to the parameter space of the pattern generator. In order to train this net, the nervous system model includes a sensitivity model that transforms the error from the arm end-point coordinates to the parameter coordinates. The error is assessed only at the termination of the movement from knowledge of the results. The role of the non-linearity in the muscle model and the performance of the learning scheme are analysed, illustrated in simulations and discussed. The results of the present study demonstrate the central nervous system’s (CNS) ability to generate typical reaching movements with a simple feedforward controller that controls only the timing and amplitude of rectangular excitation pulses to the muscles and adjusts these parameters based on knowledge of the results. In this scheme, which is based on the adjustment of only a few parameters instead of the whole trajectory, the dimension of the control problem is reduced significantly. It is shown that the non-linear properties of the muscles are essential to achieve this simple control. This conclusion agrees with the general concept that motor control is the result of an interaction between the nervous system and the musculoskeletal dynamics. Received : 21 May 1996 / Accepted in revised form : 10 June 1997     

A model of the neuro-musculo-skeletal system for human locomotion

The generation of human locomotion was examined by linking computational neuroscience with biomechanics from the perspective of nonlinear dynamical theory. We constructed a model of human locomotion, which includes a musculo-skeletal system with 8 segments and 20 muscles, a neural rhythm generator composed of 7 pairs of neural oscillators, and mechanisms for processing and transporting sensory and motor signals. Using a computer simulation, we found that locomotion emerged as a stable limit cycle that was generated by the global entrainment between the musculo-skeletal system, the neural system, and the environment. Moreover, the walking movements of the model could be compared quantitatively with those of experimental studies in humans.Part of this paper was presented to IVth International Symposium on Computer Simulation in Biomechanics, Paris, France, July 1, 1993     

Abnormalities in the awareness and control of action

Much of the functioning of the motor system occurs without awareness. Nevertheless, we are aware of some aspects of the current state of the system and we can prepare and make movements in the imagination. These mental representations of the actual and possible states of the system are based on two sources: sensory signals from skin and muscles, and the stream of motor commands that have been issued to the system. Damage to the neural substrates of the motor system can lead to abnormalities in the awareness of action as well as defects in the control of action. We provide a framework for understanding how these various abnormalities of awareness can arise. Patients with phantom limbs or with anosognosia experience the illusion that they can move their limbs. We suggest that these representations of movement are based on streams of motor commands rather than sensory signals. Patients with utilization behaviour or with delusions of control can no longer properly link their intentions to their actions. In these cases the impairment lies in the representation of intended movements. The location of the neural damage associated with these disorders suggests that representations of the current and predicted state of the motor system are in parietal cortex, while representations of intended actions are found in prefrontal and premotor cortex.     

Toward the Restoration of Hand Use to a Paralyzed Monkey: Brain-Controlled Functional Electrical Stimulation of Forearm Muscles

Loss of hand use is considered by many spinal cord injury survivors to be the most devastating consequence of their injury. Functional electrical stimulation (FES) of forearm and hand muscles has been used to provide basic, voluntary hand grasp to hundreds of human patients. Current approaches typically grade pre-programmed patterns of muscle activation using simple control signals, such as those derived from residual movement or muscle activity. However, the use of such fixed stimulation patterns limits hand function to the few tasks programmed into the controller. In contrast, we are developing a system that uses neural signals recorded from a multi-electrode array implanted in the motor cortex; this system has the potential to provide independent control of multiple muscles over a broad range of functional tasks. Two monkeys were able to use this cortically controlled FES system to control the contraction of four forearm muscles despite temporary limb paralysis. The amount of wrist force the monkeys were able to produce in a one-dimensional force tracking task was significantly increased. Furthermore, the monkeys were able to control the magnitude and time course of the force with sufficient accuracy to track visually displayed force targets at speeds reduced by only one-third to one-half of normal. Although these results were achieved by controlling only four muscles, there is no fundamental reason why the same methods could not be scaled up to control a larger number of muscles. We believe these results provide an important proof of concept that brain-controlled FES prostheses could ultimately be of great benefit to paralyzed patients with injuries in the mid-cervical spinal cord.     

Development of the trigeminal motor neurons in parrots: Implications for the role of nervous tissue in the evolution of jaw muscle morphology

Vertebrates have succeeded to inhabit almost every ecological niche due in large part to the anatomical diversification of their jaw complex. As a component of the feeding apparatus, jaw muscles carry a vital role for determining the mode of feeding. Early patterning of the jaw muscles has been attributed to cranial neural crest‐derived mesenchyme, however, much remains to be understood about the role of nonneural crest tissues in the evolution and diversification of jaw muscle morphology. In this study, we describe the development of trigeminal motor neurons in a parrot species with the uniquely shaped jaw muscles and compare its developmental pattern to that in the quail with the standard jaw muscles to uncover potential roles of nervous tissue in the evolution of vertebrate jaw muscles. In parrot embryogenesis, the motor axon bundles are detectable within the muscular tissue only after the basic shape of the muscular tissue has been established. This supports the view that nervous tissue does not primarily determine the spatial pattern of jaw muscles. In contrast, the trigeminal motor nucleus, which is composed of somata of neurons that innervate major jaw muscles, of parrot is more developed compared to quail, even in embryonic stage where no remarkable interspecific difference in both jaw muscle morphology and motor nerve branching pattern is recognized. Our data suggest that although nervous tissue may not have a large influence on initial patterning of jaw muscles, it may play an important role in subsequent growth and maintenance of muscular tissue and alterations in cranial nervous tissue development may underlie diversification of jaw muscle morphology. J. Morphol. 275:191–205, 2014. © 2013 Wiley Periodicals, Inc.     

基于鼠脑海马位置细胞与Q学习面向目标导航

生理学实验表明在鼠脑海马结构中存在的一种具有特异性放电特征的细胞,它在大鼠空间导航和环境认知中起着关键性作用,该特异性神经元被称之为位置细胞。本文将基于位置细胞、运动神经元来构建一种前馈神经网络模型,采用Q学习算法实现大鼠面向目标导航任务。实验结果表明该前馈神经网络模型能快速实现大鼠面向目标导航任务。     

A model of the cerebellar pathways applied to the control of a single-joint robot arm actuated by McKibben artificial muscles

 This article describes an expanded version of a previously proposed motor control scheme, based on rules for combining sensory and motor signals within the central nervous system. Classical control elements of the previous cybernetic circuit were replaced by artificial neural network modules having an architecture based on the connectivity of the cerebellar cortex, and whose functioning is regulated by reinforcement learning. The resulting model was then applied to the motion control of a mechanical, single-joint robot arm actuated by two McKibben artificial muscles. Various biologically plausible learning schemes were studied using both simulations and experiments. After learning, the model was able to accurately pilot the movements of the robot arm, both in velocity and position. Received: 4 September 2000 / Accepted in revised form: 7 November 2001     

Analysis and classification of compressed EMG signals by wavelet transform via alternative neural networks algorithms

We propose intelligent methods for classifying three different muscle types, i.e. biceps, frontallis and abductor pollicis brevis muscles, with low computational complexity. For this aim, electromyogram (EMG) signals are recorded and modelled by using an auto-regressive (AR) model. As the size of the EMG signals is usually large, the computational complexity of artificial neural network (ANN) systems drastically increases. Therefore, in the proposed scheme EMG signals are pre-processed by using a wavelet transform and then they are modelled by employing an AR approach. The AR coefficients are used to train and test the ANNs. Experimental results show that the highest achieved classification accuracy is more than 95% in the case of EMG signals pre-processed by wavelet transform. The wavelet transform-based pre-processing significantly increases the performance rates compared to standard multilayer perceptron and general regression neural networks algorithms.     

On the definition of central pattern generator and its sensory control

A central pattern generator (CPG) is defined here as a neural network responsible for the production of the timing cues of a rhythmic motor output pattern in the isolated CNS. For the intact animal, model considerations show that this term is neither clearly delimited from the concept of a reflex chain nor from the concept of a pattern generator with functional principles different from those of the CPG. Therefore, it cannot be concluded from the existence of a CPG in the isolated nervous system that this CPG also provides the decisive timing cues in the intact animal. Consequences for the study of the neural basis of rhythmic movements are shown.     

Motion control of musculoskeletal systems with redundancy

Motion control of musculoskeletal systems for functional electrical stimulation (FES) is a challenging problem due to the inherent complexity of the systems. These include being highly nonlinear, strongly coupled, time-varying, time-delayed, and redundant. The redundancy in particular makes it difficult to find an inverse model of the system for control purposes. We have developed a control system for multiple input multiple output (MIMO) redundant musculoskeletal systems with little prior information. The proposed method separates the steady-state properties from the dynamic properties. The dynamic control uses a steady-state inverse model and is implemented with both a PID controller for disturbance rejection and an artificial neural network (ANN) feedforward controller for fast trajectory tracking. A mechanism to control the sum of the muscle excitation levels is also included. To test the performance of the proposed control system, a two degree of freedom ankle–subtalar joint model with eight muscles was used. The simulation results show that separation of steady-state and dynamic control allow small output tracking errors for different reference trajectories such as pseudo-step, sinusoidal and filtered random signals. The proposed control method also demonstrated robustness against system parameter and controller parameter variations. A possible application of this control algorithm is FES control using multiple contact cuff electrodes where mathematical modeling is not feasible and the redundancy makes the control of dynamic movement difficult.     

Patterning of skeletal muscle

Recent studies challenge the view that signals provided by motor neurons are required to activate subsynaptic nuclei and induce postsynaptic specializations in developing skeletal muscle. New findings show that acetylcholine receptor genes are expressed and that acetylcholine receptor clusters form preferentially in the prospective synaptic region of muscle independently of motor innervation. These results indicate that developing myotubes are patterned by mechanisms intrinsic to developing muscles and raise the possibility that patterning of muscles may influence the growth pattern of motor axons and the sites where synapses form.