Computational nature of human adaptive control during learning of reaching movements in force fields

Learning to make reaching movements in force fields was used as a paradigm to explore the system architecture of the biological adaptive controller. We compared the performance of a number of candidate control systems that acted on a model of the neuromuscular system of the human arm and asked how well the dynamics of the candidate system compared with the movement characteristics of 16 subjects. We found that control via a supra-spinal system that utilized an adaptive inverse model resulted in dynamics that were similar to that observed in our subjects, but lacked essential characteristics. These characteristics pointed to a different architecture where descending commands were influenced by an adaptive forward model. However, we found that control via a forward model alone also resulted in dynamics that did not match the behavior of the human arm. We considered a third control architecture where a forward model was used in conjunction with an inverse model and found that the resulting dynamics were remarkably similar to that observed in the experimental data. The essential property of this control architecture was that it predicted a complex pattern of near-discontinuities in hand trajectory in the novel force field. A nearly identical pattern was observed in our subjects, suggesting that generation of descending motor commands was likely through a control system architecture that included both adaptive forward and inverse models. We found that as subjects learned to make reaching movements, adaptation rates for the forward and inverse models could be independently estimated and the resulting changes in performance of subjects from movement to movement could be accurately accounted for. Results suggested that the adaptation of the forward model played a dominant role in the motor learning of subjects. After a period of consolidation, the rates of adaptation in the internal models were significantly larger than those observed before the memory had consolidated. This suggested that consolidation of motor memory coincided with freeing of certain computational resources for subsequent learning. Received: 01 January 1998 / Accepted in revised form: 26 January 1999     

Impedance characteristics of a neuromusculoskeletal model of the human arm II. Movement control

The modulation of neuromusculoskeletal impedance during movements is analysed using a motor control model of the human arm. The motor control system combines feedback and feedforward control and both control modes are determined in one optimization process. In the model, the stiffness varies at the double movement frequency for 2-Hz oscillatory elbow movements and has high values at the movement reversals. During goal-directed two-degrees-of-freedom arm movements, the stiffness is decreased during the movement and may be increased in the initial and final phases, depending on the movement velocity. The stiffness has a considerable curl during the movement, as was also observed in experimental data. The dynamic stiffness patterns of the model can be explained basically by the α−γ coactivation scheme where feedback gains covary with motor control signals. In addition to the modulation of the gain factors, it is argued that the variation of the intrinsic stiffness has a considerable effect on movement control, especially during fast movements. Received: 14 October 1997 / Accepted in revised form: 18 May 1999     

A modular artificial neural net for controlling a six-legged walking system

 A system that controls the leg movement of an animal or a robot walking over irregular ground has to ensure stable support for the body and at the same time propel it forward. To do so, it has to react adaptively to unpredictable features of the environment. As part of our study of the underlying mechanisms, we present here a model for the control of the leg movement of a 6-legged walking system. The model is based on biological data obtained from the stick insect. It represents a combined treatment of realistic kinematics and biologically motivated, adaptive gait generation. The model extends a previous algorithmic model by substituting simple networks of artificial neurons for the algorithms previously used to control leg state and interleg coordination. Each system controlling an individual leg consists of three subnets. A hierarchically superior net contains two sensory and two ‘premotor’ units; it rhythmically suppresses the output of one or the other of the two subordinate nets. These are continuously active. They might be called the ‘swing module’ and the ‘stance module’ because they are responsible for controlling the swing (return stroke) and the stance (power stroke) movements, respectively. The swing module consists of three motor units and seven sensory units. It can produce appropriate return stroke movements for a broad range of initial and final positions, can cope with mechanical disturbances of the leg movement, and is able to react to an obstacle which hinders the normal performance of the swing movement. The complete model is able to walk at different speeds over irregular surfaces. The control system rapidly reestablishes a stable gait when the movement of the legs is disturbed. Received: 13 July 1994/Accepted in revised form: 15 November 1994     

Spinal Mechanisms May Provide a Combination of Intermittent and Continuous Control of Human Posture: Predictions from a Biologically Based Neuromusculoskeletal Model

Several models have been employed to study human postural control during upright quiet stance. Most have adopted an inverted pendulum approximation to the standing human and theoretical models to account for the neural feedback necessary to keep balance. The present study adds to the previous efforts in focusing more closely on modelling the physiological mechanisms of important elements associated with the control of human posture. This paper studies neuromuscular mechanisms behind upright stance control by means of a biologically based large-scale neuromusculoskeletal (NMS) model. It encompasses: i) conductance-based spinal neuron models (motor neurons and interneurons); ii) muscle proprioceptor models (spindle and Golgi tendon organ) providing sensory afferent feedback; iii) Hill-type muscle models of the leg plantar and dorsiflexors; and iv) an inverted pendulum model for the body biomechanics during upright stance. The motor neuron pools are driven by stochastic spike trains. Simulation results showed that the neuromechanical outputs generated by the NMS model resemble experimental data from subjects standing on a stable surface. Interesting findings were that: i) an intermittent pattern of muscle activation emerged from this posture control model for two of the leg muscles (Medial and Lateral Gastrocnemius); and ii) the Soleus muscle was mostly activated in a continuous manner. These results suggest that the spinal cord anatomy and neurophysiology (e.g., motor unit types, synaptic connectivities, ordered recruitment), along with the modulation of afferent activity, may account for the mixture of intermittent and continuous control that has been a subject of debate in recent studies on postural control. Another finding was the occurrence of the so-called “paradoxical” behaviour of muscle fibre lengths as a function of postural sway. The simulations confirmed previous conjectures that reciprocal inhibition is possibly contributing to this effect, but on the other hand showed that this effect may arise without any anticipatory neural control mechanism.     

Implications of different classes of sensorimotor disturbance for cerebellar-based motor learning models

The exact role of the cerebellum in motor control and learning is not yet fully understood. The structure, connectivity and plasticity within cerebellar cortex has been extensively studied, but the patterns of connectivity and interaction with other brain structures, and the computational significance of these patterns, is less well known and a matter of debate. Two contrasting models of the role of the cerebellum in motor adaptation have previously been proposed. Most commonly, the cerebellum is employed in a purely feedforward pathway, with its output contributing directly to the outgoing motor command. The cerebellum must then learn an inverse model of the motor apparatus in order to achieve accurate control. More recently, Porrill et al. (Proc Biol Sci 271(1541):789–796, 2004) and Porrill et al. (PLoS Comput Biol 3:1935–1950, 2007a) and Porrill et al. (Neural Comput 19(1), 170–193, 2007b) have highlighted the potential importance of these recurrent connections by proposing an alternative architecture in which the cerebellum is embedded in a recurrent loop with brainstem control circuitry. In this framework, the feedforward connections are not necessary at all. The cerebellum must learn a forward model of the motor apparatus for accurate motor commands to be generated. We show here how these two models exhibit contrasting yet complimentary learning capabilities. Central to the differences in performance between architectures is that there are two distinct kinds of disturbance to which a motor system may need to adapt (1) changes in the relationship between the motor command and the observed outcome and (2) changes in the relationship between the stimulus and the desired outcome. The computational distinction between these two kinds of transformation is subtle and has therefore often been overlooked. However, the implications for learning turn out to be significant: learning with a feedforward architecture is robust following changes in the stimulus-desired outcome mapping but not necessarily the motor command-outcome mapping, while learning with a recurrent architecture is robust under changes in the motor command-outcome mapping but not necessarily the stimulus-desired outcome mapping. We first analyse these differences theoretically and through simulations in the vestibulo-ocular reflex (VOR), then illustrate how these same concepts apply more generally with a model of reaching movements.     

Feedback and delays in neurological diseases: A modeling study using gynamical systems

Numerous regulatory mechanisms in motor control involve the presence of time delays in the controlled behavior of the system. Experimentally, we have shown that an increase of the time delay in visual feedback induces different oscillations in control subjects and in patients with neurological diseases during the performance of a simple compensatory tracking task. A preliminary model is proposed to describe the oscillations observed in control subjects and in patients with neurological diseases. The influence of delays in two feedback loops are the main components of the motor control circuitry involved in this task and are studied from an analytical and physiological perspective. We analytically determine the influence in the model of each of these delays on the stability of the finger position. In addition, the influence of stochastic elements (“noise”) in the modeling equation is seen to contribute qualitatively to a more accurate reproduction of experimental traces in patients with Parkinson's disease but not in patients with cerebellar disease.     

Examination of the λ equilibrium point hypothesis when applied to single degree of freedom movements performed with different inertial loads

 One of the theories of human motor control is the λ Equilibrium Point Hypothesis. It is an attractive theory since it offers an easy control scheme where the planned trajectory shifts monotionically from an initial to a final equilibrium state. The feasibility of this model was tested by reconstructing the virtual trajectory and the stiffness profiles for movements performed with different inertial loads and examining them. Three types of movements were tested: passive movements, targeted movements, and repetitive movements. Each of the movements was performed with five different inertial loads. Plausible virtual trajectories and stiffness profiles were reconstructed based on the λ Equilibrium Point Hypothesis for the three different types of movements performed with different inertial loads. However, the simple control strategy supported by the model, where the planned trajectory shifts monotonically from an initial to a final equilibrium state, could not be supported for targeted movements performed with added inertial load. To test the feasibility of the model further we must examine the probability that the human motor control system would choose a trajectory more complicated than the actual trajectory to control. Received: 20 June 1995 / Accepted in revised form: 6 August 1996     

Impedance characteristics of a neuromusculoskeletal model of the human arm I. Posture control

The mechanical impedance of neuromusculoskeletal models of the human arm is studied in this paper. The model analysis provides a better understanding of the contributions of possible intrinsic and reflexive components of arm impedance, makes clear the limitations of second-order mass-viscosity-stiffness models and reveals possible task effects on the impedance. The musculoskeletal model describes planar movements of the upper arm and forearm, which are moved by six lumped muscles with nonlinear dynamics. The motor control system is represented by a neural network which combines feedforward and feedback control. It is optimized for the control of movements or for posture control in the presence of external forces. The achieved impedance characteristics depend on the conditions during the learning process. In particular, the impedance is adapted in a suitable way to the frequency content and direction of external forces acting on the hand during an isometric task. The impedance characteristics of a model, which is optimized for movement control, are similar to experimental data in the literature. The achieved stiffness is, to a large extent, reflexively determined whereas the approximated viscosity is primarily due to intrinsic attributes. It is argued that usually applied Hill-type muscle models do not properly represent intrinsic muscle stiffness. Received: 14 October 1997 / Accepted in revised form: 18 May 1999     

Two-year-old children copy more reliably and more often than nonhuman great apes in multiple observational learning tasks

Individuals observing a proficient model can potentially benefit by copying at least one of the following three elements: motor movements (i.e., actions), goals, and results. Although several studies have investigated this issue in human infants, there are still very few studies that have systematically examined great apes’ ability to spontaneously copy each of these three elements (particularly in comparison with human infants). We tested great apes and human children with eight two-target puzzle boxes—with varying levels of difficulty—to isolate the aspects that the various species may be more prone to copying. We found first trial evidence for observational learning of actions, goals, and results in children. Some copying was found for apes as well, but only if their performance was averaged across trials.     

Neural network mosaic model for pupillary responses to spatial stimuli

A neural network mosaic model was developed to investigate the spatial-temporal properties of the human pupillary control system. It was based on the double-layer neural network model developed by Cannon and Robinson and the pupillary dual-path model developed by Sun and Stark. The neural network portion of the model received its input from a sensor array and consisted of a retina-like two-dimensional neuronal layer. The dual-path portion of the model was composed of interconnections of the neurons that formed a mosaic of AC transient and DC sustained paths. The spatial aggregates of the AC and DC signals were input to the AC and DC summing neurons, respectively. Finally, the weighted sum of the aggregate AC and DC signals provided the output for driving the pupillary response. An important property of the model was that it could adaptively learn from training samples by adjustment of the weights. The neural network mosaic model showed excellent performance in simulating both the traditional pupillary phenomena and the new spatial stimulation findings such as responses to change in stimulus pattern and shift of light spot. Moreover, the model could also be used for the diagnosis of clinical deficits and image processing in machine vision. Received: 12 December 1997 / Accepted in revised form: 22 April 1998     

Spectral properties of human cognition and skill

 Many interactive human skills are based on real-time error detection and correction. Here we investigate the spectral properties of such skills, focusing on a synchronization task. A simple autoregressive error correction model, based on separate ‘motor’ and ‘cognitive’ sources, provides an excellent fit to experimental spectral data. The model can also apply to recurrent processes not based on error correction, allowing commentary on previous claims of 1/ f-type noise in human cognition. A comparison of expert and non-expert subjects suggests that performance skill is not only based on reduced variance and bias, but also on the construction of richer mental models of error correction. Received: 4 October 1995 / Accepted in revised form: 25 February 1997     

A distributed-parameter model of the myelinated human motor nerve fibre: temporal and spatial distributions of action potentials and ionic currents

 A double cable model of the myelinated human motor nerve fibre is presented. The model is based on the nodal and internodal channels in a previous, two-component model of human motor axons (Bostock et al. 1991), added to a complex extended cable structure of nodal, paranodal and internodal segments. The model assumes a high-resistance myelin sheath and a leakage pathway to the internodal axolemma via the paranodal seal resistance and periaxonal space. The parameter values of the model were adjusted to match the recordings of threshold electrotonus in human motor fibres from Bostock et al. (1991). Kirchoff ’s current law was used to derive a system of partial differential equations for the electrical equivalent circuit, and numerical integration was performed with a fixed time increment and non-uniform spatial step sizes, in accordance with the complex structure of the fibre. The model calculations provide estimates of the spatial and temporal distributions of action potentials and their transaxonal and transmyelin components, both in different segments of the fibre and at different moments during action potential propagation. The distribution of transaxonal and transmyelin currents along the fibre and their contributions from different ionic channels are also explored. Received: 14 July 1994/Accepted in revised form: 4 April 1995     

Correlations in state space can cause sub-optimal adaptation of optimal feedback control models

Control of our movements is apparently facilitated by an adaptive internal model in the cerebellum. It was long thought that this internal model implemented an adaptive inverse model and generated motor commands, but recently many reject that idea in favor of a forward model hypothesis. In theory, the forward model predicts upcoming state during reaching movements so the motor cortex can generate appropriate motor commands. Recent computational models of this process rely on the optimal feedback control (OFC) framework of control theory. OFC is a powerful tool for describing motor control, it does not describe adaptation. Some assume that adaptation of the forward model alone could explain motor adaptation, but this is widely understood to be overly simplistic. However, an adaptive optimal controller is difficult to implement. A reasonable alternative is to allow forward model adaptation to ‘re-tune’ the controller. Our simulations show that, as expected, forward model adaptation alone does not produce optimal trajectories during reaching movements perturbed by force fields. However, they also show that re-optimizing the controller from the forward model can be sub-optimal. This is because, in a system with state correlations or redundancies, accurate prediction requires different information than optimal control. We find that adding noise to the movements that matches noise found in human data is enough to overcome this problem. However, since the state space for control of real movements is far more complex than in our simple simulations, the effects of correlations on re-adaptation of the controller from the forward model cannot be overlooked.     

A hierarchical neural-network model for control and learning of voluntary movement

In order to control voluntary movements, the central nervous system (CNS) must solve the following three computational problems at different levels: the determination of a desired trajectory in the visual coordinates, the transformation of its coordinates to the body coordinates and the generation of motor command. Based on physiological knowledge and previous models, we propose a hierarchical neural network model which accounts for the generation of motor command. In our model the association cortex provides the motor cortex with the desired trajectory in the body coordinates, where the motor command is then calculated by means of long-loop sensory feedback. Within the spinocerebellum — magnocellular red nucleus system, an internal neural model of the dynamics of the musculoskeletal system is acquired with practice, because of the heterosynaptic plasticity, while monitoring the motor command and the results of movement. Internal feedback control with this dynamical model updates the motor command by predicting a possible error of movement. Within the cerebrocerebellum — parvocellular red nucleus system, an internal neural model of the inverse-dynamics of the musculo-skeletal system is acquired while monitoring the desired trajectory and the motor command. The inverse-dynamics model substitutes for other brain regions in the complex computation of the motor command. The dynamics and the inverse-dynamics models are realized by a parallel distributed neural network, which comprises many sub-systems computing various nonlinear transformations of input signals and a neuron with heterosynaptic plasticity (that is, changes of synaptic weights are assumed proportional to a product of two kinds of synaptic inputs). Control and learning performance of the model was investigated by computer simulation, in which a robotic manipulator was used as a controlled system, with the following results: (1) Both the dynamics and the inverse-dynamics models were acquired during control of movements. (2) As motor learning proceeded, the inverse-dynamics model gradually took the place of external feedback as the main controller. Concomitantly, overall control performance became much better. (3) Once the neural network model learned to control some movement, it could control quite different and faster movements. (4) The neural netowrk model worked well even when only very limited information about the fundamental dynamical structure of the controlled system was available. Consequently, the model not only accounts for the learning and control capability of the CNS, but also provides a promising parallel-distributed control scheme for a large-scale complex object whose dynamics are only partially known.     

Dynamic updating of distributed neural representations using forward models

In this paper, we present a continuous attractor network model that we hypothesize will give some suggestion of the mechanisms underlying several neural processes such as velocity tuning to visual stimulus, sensory discrimination, sensorimotor transformations, motor control, motor imagery, and imitation. All of these processes share the fundamental characteristic of having to deal with the dynamic integration of motor and sensory variables in order to achieve accurate sensory prediction and/or discrimination. Such principles have already been described in the literature by other high-level modeling studies (Decety and Sommerville in Trends Cogn Sci 7:527–533, 2003; Oztop et al. in Neural Netw 19(3):254–271, 2006; Wolpert et al. in Philos Trans R Soc 358:593–602, 2003). With respect to these studies, our work is more concerned with biologically plausible neural dynamics at a population level. Indeed, we show that a relatively simple extension of the classical neural field models can endow these networks with additional dynamic properties for updating their internal representation using external commands. Moreover, an analysis of the interactions between our model and external inputs also shows interesting properties, which we argue are relevant for a better understanding of the neural processes of the brain.     

Fractional Calculus in Biomechanics: A 3D Viscoelastic Model Using Regularized Fractional Derivative Kernels with Application to the Human Calcaneal Fat Pad

A viscoelastic model of the K-BKZ (Kaye, Technical Report 134, College of Aeronautics, Cranfield 1962; Bernstein et al., Trans Soc Rheol 7: 391–410, 1963) type is developed for isotropic biological tissues and applied to the fat pad of the human heel. To facilitate this pursuit, a class of elastic solids is introduced through a novel strain-energy function whose elements possess strong ellipticity, and therefore lead to stable material models. This elastic potential – via the K-BKZ hypothesis – also produces the tensorial structure of the viscoelastic model. Candidate sets of functions are proposed for the elastic and viscoelastic material functions present in the model, including two functions whose origins lie in the fractional calculus. The Akaike information criterion is used to perform multi-model inference, enabling an objective selection to be made as to the best material function from within a candidate set.Dedicated to Prof. Ronald L. Bagley.     

A model of the cerebellar pathways applied to the control of a single-joint robot arm actuated by McKibben artificial muscles

 This article describes an expanded version of a previously proposed motor control scheme, based on rules for combining sensory and motor signals within the central nervous system. Classical control elements of the previous cybernetic circuit were replaced by artificial neural network modules having an architecture based on the connectivity of the cerebellar cortex, and whose functioning is regulated by reinforcement learning. The resulting model was then applied to the motion control of a mechanical, single-joint robot arm actuated by two McKibben artificial muscles. Various biologically plausible learning schemes were studied using both simulations and experiments. After learning, the model was able to accurately pilot the movements of the robot arm, both in velocity and position. Received: 4 September 2000 / Accepted in revised form: 7 November 2001     

Coupling of proton flow and rotation in the bacterial flagellar motor: stochastic simulation of a microscopic model

Summary The rotatory motor of bacterial flagella is driven by a transmembrane electrochemical gradient of protons. A model of the flagellar motor is analysed, which is based on the notion that protons passing through the motor use a channel-like pathway formed by ligand groups located partly on the rotor, partly on the stator. Proton translocation is linked to the displacement of stator elements which are elastically bound to the cell wall. The model is described by a cyclic sequence of translocation steps and proton binding and release reactions. Stochastic simulations of the model are carried out in which transitions between the states of the reaction cycle are treated as random events. In this way the rotation frequency can be predicted as a function of experimental variables such as driving force and viscous load. Furthermore, the effects of microscopic parameters such as the transition frequencies of stator elements and the force constant of elastic coupling on the dynamic properties of the motor can be studied. The model allows for intrinsic uncoupling (“slippage”) resulting from translocation steps without associated rotational movement. It is shown that mechanistic information can be obtained by studying random fluctuations of rotational speed. Offprint requests to: P. Lauger     

Manuo-ocular coordination in target tracking. I. A model simulating human performance

During eye tracking of a self-moved target, human subjects' performance differs from eye-alone tracking of an external target. Typical latency between target and eye motion onsets is shorter, ocular smooth pursuit (SP) saturation velocity increases and the maximum target motion frequency at which the SP system functions correctly is higher. Based on a previous qualitative model, a quantitative model of the coordination control between the arm motor system and the SP system is presented and evaluated here. The model structure maintains a high level of parallelism with the physiological system. It contains three main parts: the eye motor control (containing a SP branch and a saccadic branch), the arm motor control and the coordination control. The coordination control is achieved via an exchange of information between the arm and the eye sensorimotor systems, mediated by sensory signals (vision, proprioception) and motor command copy. This cross-talk results in improved SP system performance. The model has been computer simulated and the results have been compared with human subjects' behavior observed during previous experiments. The model performance is seen to quantitatively fit data on human subjects. Received: 6 March 1997 / Accepted in revised form: 15 July 1997     

Simulating a neural cross-talk model for between-hand interference during bimanual circle drawing

Studies on drawing circles with both hands in the horizontal plane have shown that this task is easy to perform across a wide range of movement frequencies under the symmetrical mode of coordination, whereas under the asymmetrical mode (both limbs moving clockwise or counterclockwise) increases in movement frequency have a disruptive effect on trajectory control and hand coordination. To account for these interference effects, we propose a simplified computer model for bimanual circle drawing based on the assumptions that (1) circular trajectories are generated from two orthogonal oscillations coupled with a phase delay, (2) the trajectories are organized on two levels, “intention” and “motor execution”, and (3) the motor systems controlling each hand are prone to neural cross-talk. The neural cross-talk consists in dispatching some fraction of any force command sent to one limb as a mirror image to the other limb. Assuming predominating coupling influences from the dominant to the nondominant limb, the simulations successfully reproduced the main characteristics of performance during asymmetrical bimanual circle drawing with increasing movement frequencies, including disruption of the circular form drawn with the nondominant hand, increasing dephasing of the hand movements, increasing variability of the phase difference, and occasional reversals of the movement direction in the nondominant limb. The implications of these results for current theories of bimanual coordination are discussed. Received: 23 June 1998 / Accepted in revised form: 20 April 1999