A model of time estimation and error feedback in predictive timing behavior

Two key features of sensorimotor prediction are preprogramming and adjusting of performance based on previous experience. Oculomotor tracking of alternating visual targets provides a simple paradigm to study this behavior in the motor system; subjects make predictive eye movements (saccades) at fast target pacing rates (>0.5 Hz). In addition, the initiation errors (latencies) during predictive tracking are correlated over a small temporal window (correlation window) suggesting that tracking performance within this time range is used in the feedback process of the timing behavior. In this paper, we propose a closed-loop model of this predictive timing. In this model, the timing between movements is based on an internal estimation of stimulus timing (an internal clock), which is represented by a (noisy) signal integrated to a threshold. The threshold of the integrate-to-fire mechanism is determined by the timing between movements made within the correlation window of previous performance and adjusted by feedback of recent and projected initiation error. The correlation window size increases with repeated tracking and was estimated by two independent experiments. We apply the model to several experimental paradigms and show that it produces data specific to predictive tracking: a gradual shift from reaction to prediction on initial tracking, phase transition and hysteresis as pacing frequency changes, scalar property, continuation of predictive tracking despite perturbations, and intertrial correlations of a specific form. These results suggest that the process underlying repetitive predictive motor timing is adjusted by the performance and the corresponding errors accrued over a limited time range and that this range increases with continued confidence in previous performance.     

Predictive feedback in human simulated pendulum balancing

In studies of human balance, it is common to fit stimulus-response data by tuning the time-delay and gain parameters of a simple delayed feedback model. Many interpret this fitted model, a simple delayed feedback model, as evidence that predictive processes are not required to explain existing data on standing balance. However, two questions lead us to doubt this approach. First, does fitting a delayed feedback model lead to reliable estimates of the time-delay? Second, can a non-predictive controller provide an explanation compatible with the independently estimated time delay? For methodological and experimental clarity, we study human balancing of a simulated inverted pendulum via joystick and screen. A two-step approach to data analysis is used: firstly a non-parametric model—the closed-loop impulse response—is estimated from the experimental data; second, a parametric model is fitted to the non-parametric impulse-response by adjusting time-delay and controller parameters. To support the second step, a new explicit formula relating controller parameters to closed-loop impulse response is derived. Two classes of controller are investigated within a common state-space context: non-predictive and predictive. It is found that the time-delay estimate arising from the second step is strongly dependent on which controller class is assumed; in particular, the non-predictive control assumption leads to time-delay estimates that are smaller than those arising from the predictive assumption. Moreover, the time-delays estimated using the non-predictive control assumption are not consistent with a lower-bound on the time-delay of the non-parametric model whereas the corresponding predictive result is consistent. Thus while the goodness of fit only marginally favoured predictive over non-predictive control, if we add the additional constraint that the model must reproduce the non-parametric time delay, then the non-predictive control model fails. We conclude (1) the time-delay should be estimated independently of fitting a low order parametric model, (2) that balance of the simulated inverted pendulum could not be explained by the non-predictive control model and (3) that predictive control provided a better explanation than non-predictive control.     

Intermittent control: a computational theory of human control

The paradigm of continuous control using internal models has advanced understanding of human motor control. However, this paradigm ignores some aspects of human control, including intermittent feedback, serial ballistic control, triggered responses and refractory periods. It is shown that event-driven intermittent control provides a framework to explain the behaviour of the human operator under a wider range of conditions than continuous control. Continuous control is included as a special case, but sampling, system matched hold, an intermittent predictor and an event trigger allow serial open-loop trajectories using intermittent feedback. The implementation here may be described as ??continuous observation, intermittent action??. Beyond explaining unimodal regulation distributions in common with continuous control, these features naturally explain refractoriness and bimodal stabilisation distributions observed in double stimulus tracking experiments and quiet standing, respectively. Moreover, given that human control systems contain significant time delays, a biological-cybernetic rationale favours intermittent over continuous control: intermittent predictive control is computationally less demanding than continuous predictive control. A standard continuous-time predictive control model of the human operator is used as the underlying design method for an event-driven intermittent controller. It is shown that when event thresholds are small and sampling is regular, the intermittent controller can masquerade as the underlying continuous-time controller and thus, under these conditions, the continuous-time and intermittent controller cannot be distinguished. This explains why the intermittent control hypothesis is consistent with the continuous control hypothesis for certain experimental conditions.     

Feedback and delays in neurological diseases: A modeling study using gynamical systems

Numerous regulatory mechanisms in motor control involve the presence of time delays in the controlled behavior of the system. Experimentally, we have shown that an increase of the time delay in visual feedback induces different oscillations in control subjects and in patients with neurological diseases during the performance of a simple compensatory tracking task. A preliminary model is proposed to describe the oscillations observed in control subjects and in patients with neurological diseases. The influence of delays in two feedback loops are the main components of the motor control circuitry involved in this task and are studied from an analytical and physiological perspective. We analytically determine the influence in the model of each of these delays on the stability of the finger position. In addition, the influence of stochastic elements (“noise”) in the modeling equation is seen to contribute qualitatively to a more accurate reproduction of experimental traces in patients with Parkinson's disease but not in patients with cerebellar disease.     

Neural model of the genetic network

Many cell control processes consist of networks of interacting elements that affect the state of each other over time. Such an arrangement resembles the principles of artificial neural networks, in which the state of a particular node depends on the combination of the states of other neurons. The lambda bacteriophage lysis/lysogeny decision circuit can be represented by such a network. It is used here as a model for testing the validity of a neural approach to the analysis of genetic networks. The model considers multigenic regulation including positive and negative feedback. It is used to simulate the dynamics of the lambda phage regulatory system; the results are compared with experimental observation. The comparison proves that the neural network model describes behavior of the system in full agreement with experiments; moreover, it predicts its function in experimentally inaccessible situations and explains the experimental observations. The application of the principles of neural networks to the cell control system leads to conclusions about the stability and redundancy of genetic networks and the cell functionality. Reverse engineering of the biochemical pathways from proteomics and DNA micro array data using the suggested neural network model is discussed.     

Integral control in accommodation.

Models of the accommodation system available inthe literature may be classified as proportional controller, optimal bang-bang controller and pure integral controller types. Each of these models explains only some aspects of the experimental behavior of the system and conflicts with other experimental observations. Based on two different experiments in which the built-in phyiological feedback loop has been opened, a leaky-integrator element is proposed for the accommodation controller. This element appears to resolve the conflict between the proportional and integral controller models and also explains the experimental observations better than any of the existing models.     

Performance limitations from delay in human and mechanical motor control

We discuss natural limitations on motor performance caused by the time delay required for feedback signals to propagate within the human body or mechanical control systems. By considering a very simple delayed linear servomechanism model, we show there exists a best possible speed-accuracy trade-off similar to Fitts' law that cannot be exceeded when delay is present. This is strictly a delay effect and does not occur for the ideal case of instantaneous feedback. We then examine the performance of the vector integration to endpoint (VITE) circuit as a model of human movement and show that when this circuit is generalized to include delayed feedback the performance may not exceed that of the servomechanism with an equal delay. We suggest the existence of such a limitation may be a ubiquitous consequence of delay in motor control with the implication that the index of performance in Fitts' law cannot arbitrarily large.     

Incorporating Prediction in Models for Two-Dimensional Smooth Pursuit

A predictive component can contribute to the command signal for smooth pursuit. This is readily demonstrated by the fact that low frequency sinusoidal target motion can be tracked with zero time delay or even with a small lead. The objective of this study was to characterize the predictive contributions to pursuit tracking more precisely by developing analytical models for predictive smooth pursuit. Subjects tracked a small target moving in two dimensions. In the simplest case, the periodic target motion was composed of the sums of two sinusoidal motions (SS), along both the horizontal and the vertical axes. Motions following the same or similar paths, but having a richer spectral composition, were produced by having the target follow the same path but at a constant speed (CS), and by combining the horizontal SS velocity with the vertical CS velocity and vice versa. Several different quantitative models were evaluated. The predictive contribution to the eye tracking command signal could be modeled as a low-pass filtered target acceleration signal with a time delay. This predictive signal, when combined with retinal image velocity at the same time delay, as in classical models for the initiation of pursuit, gave a good fit to the data. The weighting of the predictive acceleration component was different in different experimental conditions, being largest when target motion was simplest, following the SS velocity profiles.     

A quantitative synchronization model for smooth pursuit target tracking

We propose a quantitative model for human smooth pursuit tracking of a continuously moving visual target which is based on synchronization of an internal expectancy model of the target position coupled to the retinal target signal. The model predictions are tested in a smooth circular pursuit eye tracking experiment with transient target blanking of variable duration. In subjects with a high tracking accuracy, the model accounts for smooth pursuit and repeatedly reproduces quantitatively characteristic patterns of the eye dynamics during target blanking. In its simplest form, the model has only one free parameter, a coupling constant. An extended model with a second parameter, a time delay or memory term, accounts for predictive smooth pursuit eye movements which advance the target. The model constitutes an example of synchronization of a complex biological system with perceived sensory signals. Cognitive and Neurobiological Research Consortium in Traumatic Brain Injury (CNRC-TBI).     

Negative Feedback Enables Fast and Flexible Collective Decision-Making in Ants

Positive feedback plays a major role in the emergence of many collective animal behaviours. In many ants pheromone trails recruit and direct nestmate foragers to food sources. The strong positive feedback caused by trail pheromones allows fast collective responses but can compromise flexibility. Previous laboratory experiments have shown that when the environment changes, colonies are often unable to reallocate their foragers to a more rewarding food source. Here we show both experimentally, using colonies of Lasius niger, and with an agent-based simulation model, that negative feedback caused by crowding at feeding sites allows ant colonies to maintain foraging flexibility even with strong recruitment to food sources. In a constant environment, negative feedback prevents the frequently found bias towards one feeder (symmetry breaking) and leads to equal distribution of foragers. In a changing environment, negative feedback allows a colony to quickly reallocate the majority of its foragers to a superior food patch that becomes available when foraging at an inferior patch is already well underway. The model confirms these experimental findings and shows that the ability of colonies to switch to a superior food source does not require the decay of trail pheromones. Our results help to resolve inconsistencies between collective foraging patterns seen in laboratory studies and observations in the wild, and show that the simultaneous action of negative and positive feedback is important for efficient foraging in mass-recruiting insect colonies.     

Feedback delays and relaxation expectancies in EMG biofeedback

The use of noncontingent feedback controls in studies of the efficacy and process of electromyographic (EMG) biofeedback may yield results confounded by differential expectancies for relaxation. Furthermore, the role of expectancies in producing psychological and physical relaxation as well as reducing muscle activity is unclear. This study investigated the effects of feedback delays and induced relaxation expectancies on EMG activity and experienced relaxation. One hundred four non-clinical subjects participated in one auditory frontal EMG biofeedback training session. Subjects were assigned to one of four computerized feedback delay conditions (0.0037, 0.7493, 2.2481, 6.7444 s) and to one of two relaxation expectancy conditions (positive or negative). During 20 minutes of biofeedback training, all groups decreased frontal activity. Feedback delays interacted with training epochs in affecting EMG; the longest delay group reduced frontal activity more slowly than the shortest delay group during training. Positive relaxation expectancies produced greater experienced relaxation than did negative relaxation expectancies. Instrumental and expectancy factors in EMG biofeedback appear to operate independently of each other by reducing physiological activity and producing psychological relaxation respectively.     

Prefrontal neural activity when feedback is not relevant to adjust performance

It has been shown that the rostral cingulate zone (RCZ) in humans uses both positive and negative feedback to evaluate performance and to flexibly adjust behaviour. Less is known on how the feedback types are processed by the RCZ and other prefrontal brain areas, when feedback can only be used to evaluate performance, but cannot be used to adjust behaviour. The present fMRI study aimed at investigating feedback that can only be used to evaluate performance in a word-learning paradigm. One group of volunteers (N = 17) received informative, performance-dependent positive or negative feedback after each trial. Since new words had to be learnt in each trial, the feedback could not be used for task-specific adaptations. The other group (N = 17) always received non-informative feedback, providing neither information about performance nor about possible task-specific adaptations. Effects of the informational value of feedback were assessed between-subjects, comparing trials with positive and negative informative feedback to non-informative feedback. Effects of feedback valence were assessed by comparing neural activity to positive and negative feedback within the informative-feedback group. Our results show that several prefrontal regions, including the pre-SMA, the inferior frontal cortex and the insula were sensitive to both, the informational value and the valence aspect of the feedback with stronger activations to informative as compared to non-informative feedback and to informative negative compared to informative positive feedback. The only exception was RCZ which was sensitive to the informational value of the feedback, but not to feedback valence. The findings indicate that outcome information per se is sufficient to activate prefrontal brain regions, with the RCZ being the only prefrontal brain region which is equally sensitive to positive and negative feedback.     

A dichotomy for a class of cyclic delay systems

Two complementary analyses of a cyclic negative feedback system with delay are considered in this paper. The first analysis applies the work by Sontag, Angeli, Enciso and others regarding monotone control systems under negative feedback, and it implies the global attractiveness towards an equilibrium for arbitrary delays. The second one concerns the existence of a Hopf bifurcation with respect to the delay parameter, and it implies the existence of nonconstant periodic solutions for special delay values. A key idea is the use of the Schwarzian derivative, and its application for the study of Hill function nonlinearities. The positive feedback case is also addressed.     

Adding self-renewal in committed erythroid progenitors improves the biological relevance of a mathematical model of erythropoiesis

We propose a new mathematical model of erythropoiesis that takes a positive feedback of erythrocytes on progenitor apoptosis into account, and incorporates a negative feedback of erythrocytes on progenitor self-renewal. The resulting model is a system of age-structured equations that reduces to a system of delay differential equations where the delays account for progenitor compartment duration and cell cycle length. We compare this model with experimental data on an induced-anemia in mice that exhibit damped oscillations of the hematocrit before it returns to equilibrium. When we assume no self-renewal of progenitors, we obtain an inaccurate fitting of the model with experimental data. Adding self-renewal in the progenitor compartment gives better approximations, with the main features of experimental data correctly fitted. Our results indicate the importance of progenitor self-renewal in the modelling of erythropoiesis. Moreover, the model makes testable predictions on the lifespan of erythrocytes confronted to a severe anemia, and on the progenitors behavior.     

Internal noise-sustained circadian rhythms in a Drosophila model

Circadian rhythmic processes, mainly regulated by gene expression at the molecular level, have inherent stochasticity. Their robustness or resistance to internal noise has been extensively investigated by most of the previous studies. This work focuses on the constructive roles of internal noise in a reduced Drosophila model, which incorporates negative and positive feedback loops, each with a time delay. It is shown that internal noise sustains reliable oscillations with periods close to 24 h in a region of parameter space, where the deterministic kinetics would evolve to a stable steady state. The amplitudes of noise-sustained oscillations are significantly affected by the variation of internal noise level, and the best performance of the oscillations could be found at an optimal noise intensity, indicating the occurrence of intrinsic coherence resonance. In the oscillatory region of the deterministic model, the coherence of noisy circadian oscillations is suppressed by internal noise, while the period remains nearly constant over a large range of noise intensity, demonstrating robustness of the Drosophila model for circadian rhythms to intrinsic noise. In addition, the effects of time delay in the positive feedback on the oscillations are also investigated. It is found that the time delay could efficiently tune the performance of the noise-sustained oscillations. These results might aid understanding of the exploitation of intracellular noise in biochemical and genetic regulatory systems.     

Uncertainty–guided learning with scaled prediction errors in the basal ganglia

To accurately predict rewards associated with states or actions, the variability of observations has to be taken into account. In particular, when the observations are noisy, the individual rewards should have less influence on tracking of average reward, and the estimate of the mean reward should be updated to a smaller extent after each observation. However, it is not known how the magnitude of the observation noise might be tracked and used to control prediction updates in the brain reward system. Here, we introduce a new model that uses simple, tractable learning rules that track the mean and standard deviation of reward, and leverages prediction errors scaled by uncertainty as the central feedback signal. We show that the new model has an advantage over conventional reinforcement learning models in a value tracking task, and approaches a theoretic limit of performance provided by the Kalman filter. Further, we propose a possible biological implementation of the model in the basal ganglia circuit. In the proposed network, dopaminergic neurons encode reward prediction errors scaled by standard deviation of rewards. We show that such scaling may arise if the striatal neurons learn the standard deviation of rewards and modulate the activity of dopaminergic neurons. The model is consistent with experimental findings concerning dopamine prediction error scaling relative to reward magnitude, and with many features of striatal plasticity. Our results span across the levels of implementation, algorithm, and computation, and might have important implications for understanding the dopaminergic prediction error signal and its relation to adaptive and effective learning.