Cerebellar learning of accurate predictive control for fast-reaching movements

Long conduction delays in the nervous system prevent the accurate control of movements by feedback control alone. We present a new, biologically plausible cerebellar model to study how fast arm movements can be executed in spite of these delays. To provide a realistic test-bed of the cerebellar neural model, we embed the cerebellar network in a simulated biological motor system comprising a spinal cord model and a six-muscle two-dimensional arm model. We argue that if the trajectory errors are detected at the spinal cord level, memory traces in the cerebellum can solve the temporal mismatch problem between efferent motor commands and delayed error signals. Moreover, learning is made stable by the inclusion of the cerebello-nucleo-olivary loop in the model. It is shown that the cerebellar network implements a nonlinear predictive regulator by learning part of the inverse dynamics of the plant and spinal circuit. After learning, fast accurate reaching movements can be generated. Received: 8 February 1999 /Accepted in revised form: 7 August 1999     

Feedback and delays in neurological diseases: A modeling study using gynamical systems

Numerous regulatory mechanisms in motor control involve the presence of time delays in the controlled behavior of the system. Experimentally, we have shown that an increase of the time delay in visual feedback induces different oscillations in control subjects and in patients with neurological diseases during the performance of a simple compensatory tracking task. A preliminary model is proposed to describe the oscillations observed in control subjects and in patients with neurological diseases. The influence of delays in two feedback loops are the main components of the motor control circuitry involved in this task and are studied from an analytical and physiological perspective. We analytically determine the influence in the model of each of these delays on the stability of the finger position. In addition, the influence of stochastic elements (“noise”) in the modeling equation is seen to contribute qualitatively to a more accurate reproduction of experimental traces in patients with Parkinson's disease but not in patients with cerebellar disease.     

Adaptive, fast walking in a biped robot under neuronal control and learning

Human walking is a dynamic, partly self-stabilizing process relying on the interaction of the biomechanical design with its neuronal control. The coordination of this process is a very difficult problem, and it has been suggested that it involves a hierarchy of levels, where the lower ones, e.g., interactions between muscles and the spinal cord, are largely autonomous, and where higher level control (e.g., cortical) arises only pointwise, as needed. This requires an architecture of several nested, sensori–motor loops where the walking process provides feedback signals to the walker's sensory systems, which can be used to coordinate its movements. To complicate the situation, at a maximal walking speed of more than four leg-lengths per second, the cycle period available to coordinate all these loops is rather short. In this study we present a planar biped robot, which uses the design principle of nested loops to combine the self-stabilizing properties of its biomechanical design with several levels of neuronal control. Specifically, we show how to adapt control by including online learning mechanisms based on simulated synaptic plasticity. This robot can walk with a high speed (>3.0 leg length/s), self-adapting to minor disturbances, and reacting in a robust way to abruptly induced gait changes. At the same time, it can learn walking on different terrains, requiring only few learning experiences. This study shows that the tight coupling of physical with neuronal control, guided by sensory feedback from the walking pattern itself, combined with synaptic learning may be a way forward to better understand and solve coordination problems in other complex motor tasks.     

Computational mechanisms of sensorimotor control

In order to generate skilled and efficient actions, the motor system must find solutions to several problems inherent in sensorimotor control, including nonlinearity, nonstationarity, delays, redundancy, uncertainty, and noise. We review these problems and five computational mechanisms that the brain may use to limit their deleterious effects: optimal feedback control, impedance control, predictive control, Bayesian decision theory, and sensorimotor learning. Together, these computational mechanisms allow skilled and fluent sensorimotor behavior.     

Correlations in state space can cause sub-optimal adaptation of optimal feedback control models

Control of our movements is apparently facilitated by an adaptive internal model in the cerebellum. It was long thought that this internal model implemented an adaptive inverse model and generated motor commands, but recently many reject that idea in favor of a forward model hypothesis. In theory, the forward model predicts upcoming state during reaching movements so the motor cortex can generate appropriate motor commands. Recent computational models of this process rely on the optimal feedback control (OFC) framework of control theory. OFC is a powerful tool for describing motor control, it does not describe adaptation. Some assume that adaptation of the forward model alone could explain motor adaptation, but this is widely understood to be overly simplistic. However, an adaptive optimal controller is difficult to implement. A reasonable alternative is to allow forward model adaptation to ‘re-tune’ the controller. Our simulations show that, as expected, forward model adaptation alone does not produce optimal trajectories during reaching movements perturbed by force fields. However, they also show that re-optimizing the controller from the forward model can be sub-optimal. This is because, in a system with state correlations or redundancies, accurate prediction requires different information than optimal control. We find that adding noise to the movements that matches noise found in human data is enough to overcome this problem. However, since the state space for control of real movements is far more complex than in our simple simulations, the effects of correlations on re-adaptation of the controller from the forward model cannot be overlooked.     

Failure of Delayed Feedback Deep Brain Stimulation for Intermittent Pathological Synchronization in Parkinson’s Disease

Suppression of excessively synchronous beta-band oscillatory activity in the brain is believed to suppress hypokinetic motor symptoms of Parkinson’s disease. Recently, a lot of interest has been devoted to desynchronizing delayed feedback deep brain stimulation (DBS). This type of synchrony control was shown to destabilize the synchronized state in networks of simple model oscillators as well as in networks of coupled model neurons. However, the dynamics of the neural activity in Parkinson’s disease exhibits complex intermittent synchronous patterns, far from the idealized synchronous dynamics used to study the delayed feedback stimulation. This study explores the action of delayed feedback stimulation on partially synchronized oscillatory dynamics, similar to what one observes experimentally in parkinsonian patients. We employ a computational model of the basal ganglia networks which reproduces experimentally observed fine temporal structure of the synchronous dynamics. When the parameters of our model are such that the synchrony is unphysiologically strong, the feedback exerts a desynchronizing action. However, when the network is tuned to reproduce the highly variable temporal patterns observed experimentally, the same kind of delayed feedback may actually increase the synchrony. As network parameters are changed from the range which produces complete synchrony to those favoring less synchronous dynamics, desynchronizing delayed feedback may gradually turn into synchronizing stimulation. This suggests that delayed feedback DBS in Parkinson’s disease may boost rather than suppress synchronization and is unlikely to be clinically successful. The study also indicates that delayed feedback stimulation may not necessarily exhibit a desynchronization effect when acting on a physiologically realistic partially synchronous dynamics, and provides an example of how to estimate the stimulation effect.     

Estimation of psychomotor delay from the Fitts’ law coefficients

An intrinsic property of human motor behavior is a trade-off between speed and accuracy. This is classically described by Fitts’ law, a model derived by assuming that the human body has a limited capacity to transmit information in organizing motor behavior. However, Fitts’ law can also be realized as an emergent property of movements generated by delayed feedback. In this article, we describe the relationship between the Fitts’ law coefficients and the physiological parameters of the underlying delayed feedback circuit: the relaxation rate or time constant, and the psychomotor delay of the feedback process. This relationship is then used to estimate the motor circuit delay of several tasks for which Fitts’ law data are available in the literature. We consistently estimate the delay to be between 0 and 112 ms. A further consequence of this model is that not all combinations of slope and Y-intercept in Fitts’ law are possible when movements are generated by delayed feedback. In fact, it is only possible for an observed speed–accuracy trade-off to be generated by delayed feedback if the Fitts’ law coefficients satisfy ?0.482 ≤ a/b ≤ 3.343 [bits] where b represents the slope in bits per second and a represents the Y-intercept in seconds. If we assume human movement is generated by delayed feedback, then the Fitts’ law coefficients should always be restricted to this range of values.     

Adaptation to visual feedback delay influences visuomotor learning

Computational theory of motor control suggests that the brain continuously monitors motor commands, to predict their sensory consequences before actual sensory feedback becomes available. Such prediction error is a driving force of motor learning, and therefore appropriate associations between motor commands and delayed sensory feedback signals are crucial. Indeed, artificially introduced delays in visual feedback have been reported to degrade motor learning. However, considering our perceptual ability to causally bind our own actions with sensory feedback, demonstrated by the decrease in the perceived time delay following repeated exposure to an artificial delay, we hypothesized that such perceptual binding might alleviate deficits of motor learning associated with delayed visual feedback. Here, we evaluated this hypothesis by investigating the ability of human participants to adapt their reaching movements in response to a novel visuomotor environment with 3 visual feedback conditions--no-delay, sudden-delay, and adapted-delay. To introduce novelty into the trials, the cursor position, which originally indicated the hand position in baseline trials, was rotated around the starting position. In contrast to the no-delay condition, a 200-ms delay was artificially introduced between the cursor and hand positions during the presence of visual rotation (sudden-delay condition), or before the application of visual rotation (adapted-delay condition). We compared the learning rate (representing how the movement error modifies the movement direction in the subsequent trial) between the 3 conditions. In comparison with the no-delay condition, the learning rate was significantly degraded for the sudden-delay condition. However, this degradation was significantly alleviated by prior exposure to the delay (adapted-delay condition). Our data indicate the importance of appropriate temporal associations between motor commands and sensory feedback in visuomotor learning. Moreover, they suggest that the brain is able to account for such temporal associations in a flexible manner.     

A multisensory integration model of human stance control

A model is presented to study and quantify the contribution of all available sensory information to human standing based on optimal estimation theory. In the model, delayed sensory information is integrated in such a way that a best estimate of body orientation is obtained. The model approach agrees with the present theory of the goal of human balance control. The model is not based on purely inverted pendulum body dynamics, but rather on a three-link segment model of a standing human on a movable support base. In addition, the model is non-linear and explicitly addresses the problem of multisensory integration and neural time delays. A predictive element is included in the controller to compensate for time delays, necessary to maintain erect body orientation. Model results of sensory perturbations on total body sway closely resemble experimental results. Despite internal and external perturbations, the controller is able to stabilise the model of an inherently unstable standing human with neural time delays of 100 ms. It is concluded, that the model is capable of studying and quantifying multisensory integration in human stance control. We aim to apply the model in (1) the design and development of prostheses and orthoses and (2) the diagnosis of neurological balance disorders. Received: 25 August 1997 / Accepted in revised form: 8 December 1998     

The Dynamics of Turing Patterns for Morphogen-Regulated Growing Domains with Cellular Response Delays

Since its conception in 1952, the Turing paradigm for pattern formation has been the subject of numerous theoretical investigations. Experimentally, this mechanism was first demonstrated in chemical reactions over 20 years ago and, more recently, several examples of biological self-organisation have also been implicated as Turing systems. One criticism of the Turing model is its lack of robustness, not only with respect to fluctuations in the initial conditions, but also with respect to the inclusion of delays in critical feedback processes such as gene expression. In this work we investigate the possibilities for Turing patterns on growing domains where the morphogens additionally regulate domain growth, incorporating delays in the feedback between signalling and domain growth, as well as gene expression. We present results for the proto-typical Schnakenberg and Gierer–Meinhardt systems: exploring the dynamics of these systems suggests a reconsideration of the basic Turing mechanism for pattern formation on morphogen-regulated growing domains as well as highlighting when feedback delays on domain growth are important for pattern formation.     

Motor unit recruitment for dynamic tasks: current understanding and future directions

Skeletal muscle contains many muscle fibres that are functionally grouped into motor units. For any motor task there are many possible combinations of motor units that could be recruited and it has been proposed that a simple rule, the ‘size principle’, governs the selection of motor units recruited for different contractions. Motor units can be characterised by their different contractile, energetic and fatigue properties and it is important that the selection of motor units recruited for given movements allows units with the appropriate properties to be activated. Here we review what is currently understood about motor unit recruitment patterns, and assess how different recruitment patterns are more or less appropriate for different movement tasks. During natural movements the motor unit recruitment patterns vary (not always holding to the size principle) and it is proposed that motor unit recruitment is likely related to the mechanical function of the muscles. Many factors such as mechanics, sensory feedback, and central control influence recruitment patterns and consequently an integrative approach (rather than reductionist) is required to understand how recruitment is controlled during different movement tasks. Currently, the best way to achieve this is through in vivo studies that relate recruitment to mechanics and behaviour. Various methods for determining motor unit recruitment patterns are discussed, in particular the recent wavelet-analysis approaches that have allowed motor unit recruitment to be assessed during natural movements. Directions for future studies into motor recruitment within and between functional task groups and muscle compartments are suggested.     

Delayed feedback and multiple attractors in a host–parasitoid system

 Continuous-time, age structured, host–parasitoid models exhibit three types of cyclic dynamics: Lotka–Volterra-like consumer-resource cycles, discrete generation cycles, and “delayed feedback cycles” that occur if the gain to the parasitoid population (defined by the number of new female parasitoid offspring produced per host attacked) increases with the age of the host attacked. The delayed feedback comes about in the following way: an increase in the instantaneous density of searching female parasitoids increases the mortality rate on younger hosts, which reduces the density of future older and more productive hosts, and hence reduces the future per head recruitment rate of searching female parasitoids. Delayed feedback cycles have previously been found in studies that assume a step-function for the gain function. Here, we formulate a general host–parasitoid model with an arbitrary gain function, and show that stable, delayed feedback cycles are a general phenomenon, occurring with a wide range of gain functions, and strongest when the gain is an accelerating function of host age. We show by examples that locally stable, delayed feedback cycles commonly occur with parameter values that also yield a single, locally stable equilibrium, and hence their occurrence depends on initial conditions. A simplified model reveals that the mechanism responsible for the delayed feedback cycles in our host–parasitoid models is similar to that producing cycles and initial-condition-dependent dynamics in a single species model with age-dependent cannibalism. Received: 24 October 1997 / Revised version: 13 June 1998     

On the Delayed Ross–Macdonald Model for Malaria Transmission

The feedback dynamics from mosquito to human and back to mosquito involve considerable time delays due to the incubation periods of the parasites. In this paper, taking explicit account of the incubation periods of parasites within the human and the mosquito, we first propose a delayed Ross–Macdonald model. Then we calculate the basic reproduction number R ₀ and carry out some sensitivity analysis of R ₀ on the incubation periods, that is, to study the effect of time delays on the basic reproduction number. It is shown that the basic reproduction number is a decreasing function of both time delays. Thus, prolonging the incubation periods in either humans or mosquitos (via medicine or control measures) could reduce the prevalence of infection. S. Ruan’s research was partially supported by NIH grants P20RR020770-02 and R01GM083607-01, NSF grant DMS-0715772. D. Xiao’s research was supported by the National Natural Science Fund (NNSF) of China.     

The discontinuous nature of motor execution

Human movement control requires adequate coordination of different movements, which is particularly important when different motor tasks are simultaneously executed by the same effector(s) (e.g. a muscle or a joint). The process of movement execution involves a series of highly nonlinear elements; for instance, a motor unit of a muscle produces force only in the direction of muscle shortening, thus representing a threshold operator that transforms the bipolar (i.e. excitatory or inhibitory) information at its spinal input into a purely unipolar signal (i.e. muscle force). This tripartite research report addresses the contribution of the nonlinearity of neuromuscular elements to the coordination of different motor tasks simultaneously executed by the same limb. In this first part of the series, a new hypothesis for such a single-muscle multiple-task coordination is presented which suggests an essentially threshold-linear coordination mechanism. Control signals generated by the central nervous system for each individual movement independently and feedback information from peripheral receptors are linearly superimposed. This compound control/feedback signal is processed by a nonlinear limiter element reflecting the discontinuous properties of the muscle and its reflex circuitry. It is shown that threshold-linear interaction of descending commands and afferent feedback information can lead to complex interdependent patterns of compound motor action. This includes the possibility of gating (i.e. the ability of one movement pattern to constrain or even impede the execution of another pattern) and of delayed response initiation when simultaneously performing more than one voluntary motor task. A theoretical analysis of the threshold-linear coordination mechanism and an extensive experimental validation of the model is provided in part II and part III of the report. Received: 6 October 1998 / Accepted in revised form: 2 June 1999     

Impedance characteristics of a neuromusculoskeletal model of the human arm II. Movement control

The modulation of neuromusculoskeletal impedance during movements is analysed using a motor control model of the human arm. The motor control system combines feedback and feedforward control and both control modes are determined in one optimization process. In the model, the stiffness varies at the double movement frequency for 2-Hz oscillatory elbow movements and has high values at the movement reversals. During goal-directed two-degrees-of-freedom arm movements, the stiffness is decreased during the movement and may be increased in the initial and final phases, depending on the movement velocity. The stiffness has a considerable curl during the movement, as was also observed in experimental data. The dynamic stiffness patterns of the model can be explained basically by the α−γ coactivation scheme where feedback gains covary with motor control signals. In addition to the modulation of the gain factors, it is argued that the variation of the intrinsic stiffness has a considerable effect on movement control, especially during fast movements. Received: 14 October 1997 / Accepted in revised form: 18 May 1999     

Absolute stability in predator-prey models

An equilibrium of a time-lagged population model is said to be absolutely stable if it remains locally stable regardless of the length of the time delay, and it is argued that the criteria for absolute stability provide a valuable guide to the behavior of population models. For example, it is sometimes assumed that time delays have a limited impact until they exceed the natural time scale of a system; here it is stressed that under some conditions very short time delays can have a marked (and often maximal) destabilizing effect. Consequently it is important that our understanding of population dynamics is robust to the inclusion of the short time delays present in all biological systems. The absolute stability criteria are ideally suited for this role. Another important reason for using the criteria for absolute stability rather than using criteria which depend upon the details of a time delay is that biological time delays are unlikely to be constant. For example, a time delay due to maturation inevitably varies between individuals and the mean may itself vary over time. Here it is shown that the criteria for absolute stability are generally robust in the presence of distributed delays and of varying delays. The analysis presented is based upon a general predator-prey model and it is shown that absolute stability can be expected under a broad range of parameter values whenever the time delay is due to the maturation time of either the predator or the prey or of both. This stability occurs because of the interaction between delayed and undelayed dynamic features of the model. A time-delayed process, when viewed across all possible delays, always reduces stability and this effect occurs regardless of whether the process would act to stabilize or destabilize an undelayed system. Opposing the destabilization due to a time delay and making absolute stability a possibility are a number of processes which act without delay. Some of these processes can be identified as stabilizing from the analysis of undelayed models (for example, the type 3 functional response) but other cannot (for example, the nonreproductive numerical response of predators).     

Performance limitations from delay in human and mechanical motor control

We discuss natural limitations on motor performance caused by the time delay required for feedback signals to propagate within the human body or mechanical control systems. By considering a very simple delayed linear servomechanism model, we show there exists a best possible speed-accuracy trade-off similar to Fitts' law that cannot be exceeded when delay is present. This is strictly a delay effect and does not occur for the ideal case of instantaneous feedback. We then examine the performance of the vector integration to endpoint (VITE) circuit as a model of human movement and show that when this circuit is generalized to include delayed feedback the performance may not exceed that of the servomechanism with an equal delay. We suggest the existence of such a limitation may be a ubiquitous consequence of delay in motor control with the implication that the index of performance in Fitts' law cannot arbitrarily large.     

The Evolution of the Motor Control of Feeding in Amphibians

Based on studies of a few model taxa, amphibians have been consideredstereotyped in their feeding movements relative to other vertebrates.However, recent studies on a wide variety of amphibian specieshave revealed great diversity in feeding mechanics and kinematics,and illustrate that stereotypy is the exception rather thanthe rule in amphibian feeding. Apparent stereotypy in some taxamay be an artifact of unnatural laboratory conditions. The commonancestor of lissamphibians was probably capable of some modulationof feeding movements, and descendants have evolved along twotrajectories with regard to motor control: (1) an increase inmodulation via feedback or feed-forward mechanisms, as exemplifiedby ballistic-tongued plethodontid salamanders and hydrostatic-tonguedfrogs, and (2) a decrease in variation dictated by biomechanicsthat require tight coordination between different body parts,such as the tongue and jaws in toads and other frogs with ballistictongue projection. Multi-joint coordination of rapid movementsmay hamper accurate tongue placement in ballistic-tongued frogsas compared to both short-tongued frogs and ballistic tongued-salamandersthat face simpler motor control tasks. Decoupling of tongueand jaw movements is associated with increased accuracy in bothhydrostatic-tongued frogs and ballistic-tongued salamanders.     

Feedback-mediated dynamics in two coupled nephrons

Previously, we developed a dynamic model for the tubuloglomerular feedback (TGF) system in a single, short-looped nephron of the mammalian kidney. In that model, a semi-linear hyperbolic partial differential equation was used to represent two fundamental processes of solute transport in the nephron’s thick ascending limb (TAL): chloride advection by fluid flow along the TAL lumen and transepithelial chloride transport from the lumen to the interstitium. An empirical function and a time delay were used to relate glomerular filtration rate to the chloride concentration at the macula densa of the TAL. Analysis of the model equations indicated that stable limit-cycle oscillations (LCO) in nephron fluid flow and chloride concentration can emerge for sufficiently large feedback gain magnitude and time delay. In this study, the single-nephron model was extended to two nephrons, which were coupled through their filtration rates. Explicit analytical conditions were obtained for bifurcation loci corresponding to two special cases: (1) identical time delays but differing feedback gains, and (2) identical gains but differing delays. Similar to the case of a single nephron, our analysis indicates that stable LCO can emerge in coupled nephrons for sufficiently large gains and delays. However, these LCO may emerge at lower values of the feedback gain, relative to a single (i.e., uncoupled) nephron, or at shorter delays, provided the delays are sufficiently close. These results suggest that, in vivo, if two nephrons are sufficiently similar, then coupling will tend to increase the likelihood of LCO.     

具有脉冲生育和脉冲收获的时滞SEI害虫治理模型的动力学分析

根据控制害虫的生物策略,考虑了一个不同时刻害虫具有脉冲生育和对害虫进行脉冲收获的时滞的SEI害虫治理模型.我们证明了系统所有的解都是一致有界的,同时获得了无病周期解是全局吸引的充分条件.进一步,得到了具有时滞的系统持续生存的充分条件.基于研究所得到的结果,作者提出了害虫治理的一些合理建议.