Computational nature of human adaptive control during learning of reaching movements in force fields

Learning to make reaching movements in force fields was used as a paradigm to explore the system architecture of the biological adaptive controller. We compared the performance of a number of candidate control systems that acted on a model of the neuromuscular system of the human arm and asked how well the dynamics of the candidate system compared with the movement characteristics of 16 subjects. We found that control via a supra-spinal system that utilized an adaptive inverse model resulted in dynamics that were similar to that observed in our subjects, but lacked essential characteristics. These characteristics pointed to a different architecture where descending commands were influenced by an adaptive forward model. However, we found that control via a forward model alone also resulted in dynamics that did not match the behavior of the human arm. We considered a third control architecture where a forward model was used in conjunction with an inverse model and found that the resulting dynamics were remarkably similar to that observed in the experimental data. The essential property of this control architecture was that it predicted a complex pattern of near-discontinuities in hand trajectory in the novel force field. A nearly identical pattern was observed in our subjects, suggesting that generation of descending motor commands was likely through a control system architecture that included both adaptive forward and inverse models. We found that as subjects learned to make reaching movements, adaptation rates for the forward and inverse models could be independently estimated and the resulting changes in performance of subjects from movement to movement could be accurately accounted for. Results suggested that the adaptation of the forward model played a dominant role in the motor learning of subjects. After a period of consolidation, the rates of adaptation in the internal models were significantly larger than those observed before the memory had consolidated. This suggested that consolidation of motor memory coincided with freeing of certain computational resources for subsequent learning. Received: 01 January 1998 / Accepted in revised form: 26 January 1999     

Retrograde adaptive resonance theory based on the role of nitric oxide in long-term potentiation

Adaptive resonance theory (ART) demonstrates how the brain learns to recognize and categorize vast amounts of information by using top–down expectations and attentional focusing. ART 3, one member of the ART family, embeds the computational properties of the chemical synapse in its search process, but it converges slowly and is lack of stability when being applied in pattern recognition and analysis. To overcome these problems, Nitric Oxide (NO), which serves as a newly discovered retrograde messenger in Long-Term Potentiation (LTP), is introduced in retrograde adaptive resonance theory (ReART) model presented in this paper. In the presented model a novel search hypothesis is proposed to incorporate angle and amplitude information of an external input vector to decide whether the input matches the long-term memory (LTM) weights of an active node or not, and the embedded NO retrograde mechanism makes the search procedure a closed loop, which improves the stability and convergence speed of the transmitter releasing mechanism in a synapse. To make the model more adaptive and practical, a forgetting mechanism is built to improve the weights updating process. Experimental results indicate that the proposed ReART model achieves low error rate, fast convergence and self-organizing weights regulation. Action Editor: Christiane Linster     

Music Attenuates Excessive Visual Guidance of Skilled Reaching in Advanced but Not Mild Parkinson's Disease

Parkinson''s disease (PD) results in movement and sensory impairments that can be reduced by familiar music. At present, it is unclear whether the beneficial effects of music are limited to lessening the bradykinesia of whole body movement or whether beneficial effects also extend to skilled movements of PD subjects. This question was addressed in the present study in which control and PD subjects were given a skilled reaching task that was performed with and without accompanying preferred musical pieces. Eye movements and limb use were monitored with biomechanical measures and limb movements were additionally assessed using a previously described movement element scoring system. Preferred musical pieces did not lessen limb and hand movement impairments as assessed with either the biomechanical measures or movement element scoring. Nevertheless, the PD patients with more severe motor symptoms as assessed by Hoehn and Yahr (HY) scores displayed enhanced visual engagement of the target and this impairment was reduced during trials performed in association with accompanying preferred musical pieces. The results are discussed in relation to the idea that preferred musical pieces, although not generally beneficial in lessening skilled reaching impairments, may normalize the balance between visual and proprioceptive guidance of skilled reaching.     

The binding of learning to action in motor adaptation

In motor tasks, errors between planned and actual movements generally result in adaptive changes which reduce the occurrence of similar errors in the future. It has commonly been assumed that the motor adaptation arising from an error occurring on a particular movement is specifically associated with the motion that was planned. Here we show that this is not the case. Instead, we demonstrate the binding of the adaptation arising from an error on a particular trial to the motion experienced on that same trial. The formation of this association means that future movements planned to resemble the motion experienced on a given trial benefit maximally from the adaptation arising from it. This reflects the idea that actual rather than planned motions are assigned 'credit' for motor errors because, in a computational sense, the maximal adaptive response would be associated with the condition credited with the error. We studied this process by examining the patterns of generalization associated with motor adaptation to novel dynamic environments during reaching arm movements in humans. We found that these patterns consistently matched those predicted by adaptation associated with the actual rather than the planned motion, with maximal generalization observed where actual motions were clustered. We followed up these findings by showing that a novel training procedure designed to leverage this newfound understanding of the binding of learning to action, can improve adaptation rates by greater than 50%. Our results provide a mechanistic framework for understanding the effects of partial assistance and error augmentation during neurologic rehabilitation, and they suggest ways to optimize their use.     

Learning the dynamics of reaching movements results in the modification of arm impedance and long-latency perturbation responses

 Some characteristics of arm movements that humans exhibit during learning the dynamics of reaching are consistent with a theoretical framework where training results in motor commands that are gradually modified to predict and compensate for novel forces that may act on the hand. As a first approximation, the motor control system behaves as an adapting controller that learns an internal model of the dynamics of the task. It approximates inverse dynamics and predicts motor commands that are appropriate for a desired limb trajectory. However, we had previously noted that subtle motion characteristics observed during changes in task dynamics challenged this simple model and raised the possibility that adaptation also involved sensory–motor feedback pathways. These pathways reacted to sensory feedback during the course of the movement. Here we hypothesize that adaptation to dynamics might also involve a modification of how the CNS responds to sensory feedback. We tested this through experiments that quantified how the motor system's response to errors during voluntary movements changed as it adapted to dynamics of a force field. We describe a nonlinear approach that approximates the impedance of the arm, i.e., force response as a function of arm displacement trajectory. We observe that after adaptation, the impedance function changes in a way that closely matches and counters the effect of the force field. This is particularly prominent in the long-latency (>100 ms) component of response to perturbations. Therefore, it appears that practice not only modifies the internal model with which the brain generates motor commands that initiate a movement, but also the internal model with which sensory feedback is integrated with the ongoing descending commands in order to respond to error during the movement. Received: 10 January 2001 / Accepted in revised form: 30 May 2001     

Correlations in state space can cause sub-optimal adaptation of optimal feedback control models

Control of our movements is apparently facilitated by an adaptive internal model in the cerebellum. It was long thought that this internal model implemented an adaptive inverse model and generated motor commands, but recently many reject that idea in favor of a forward model hypothesis. In theory, the forward model predicts upcoming state during reaching movements so the motor cortex can generate appropriate motor commands. Recent computational models of this process rely on the optimal feedback control (OFC) framework of control theory. OFC is a powerful tool for describing motor control, it does not describe adaptation. Some assume that adaptation of the forward model alone could explain motor adaptation, but this is widely understood to be overly simplistic. However, an adaptive optimal controller is difficult to implement. A reasonable alternative is to allow forward model adaptation to ‘re-tune’ the controller. Our simulations show that, as expected, forward model adaptation alone does not produce optimal trajectories during reaching movements perturbed by force fields. However, they also show that re-optimizing the controller from the forward model can be sub-optimal. This is because, in a system with state correlations or redundancies, accurate prediction requires different information than optimal control. We find that adding noise to the movements that matches noise found in human data is enough to overcome this problem. However, since the state space for control of real movements is far more complex than in our simple simulations, the effects of correlations on re-adaptation of the controller from the forward model cannot be overlooked.     

The activity of leech motoneurons during motor patterns is regulated by intrinsic properties and synaptic inputs

The activity of motoneurons during motor patterns depends on their intrinsic properties and on synaptic inputs. This study analyzed the properties of two leech motoneurons: the excitors of dorsal longitudinal muscles (DE-3) and of dorsal and ventral longitudinal muscles (MN-L) in basal conditions (normal and high Mg²⁺ saline) and during crawling. The voltage–current relationships in DE-3 and MN-L were similar. The curves exhibited the largest slope around resting potential, showed marked inward and outward rectification, and were not affected by high Mg²⁺. In response to 5-s pulses, DE-3 exhibited a fast initial adaptation, a slow recovery and a very slow late adaptation. High Mg²⁺ abolished the initial high frequency. The frequency–voltage relationship for the rest of the response was highly similar in normal and in high Mg²⁺ saline. MN-L exhibited a minor initial adaptation and then fired steadily. High Mg²⁺ diminished the frequency–voltage relationship. During crawling DE-3 and MN-L fired in phase and their frequency–voltage curves overlapped with the lower end of the curves obtained in basal conditions. The results suggest that the activity of these motoneurons during crawling was regulated, to a large extent, by synaptic inputs.     

A model of the cerebellum in adaptive control of saccadic gain

 We review data showing that the cerebellum is required for adaptation of saccadic gain to repeated presentations of dual-step visual targets and thus, presumably, for providing adaptive corrections for the brainstem saccade generator in response to any error created by the open-loop saccadic system. We model the adaptability of the system in terms of plasticity of synapses from parallel fibers to Purkinje cells in cerebellar cortex, stressing the integration of cerebellar cortex and nuclei in microzones as the units for correction of motor pattern generators. We propose a model of the inferior olive as an error detector, and use a ‘window of eligibility’ to insure that error signals that elicit a corrective movement are used to adjust the original movement, not the secondary movement. In a companion paper we simulate this large, realistic network of neural-like units to study the complex spatiotemporal behavior of neuronal subpopulations implicated in the control and adaptation of saccades. Received: 25 November 1994/Accepted in revised form: 6 February 1996     

Single Medial Prefrontal Neurons Cope with Error

Learning from mistakes is a key feature of human behavior. However, the mechanisms underlying short-term adaptation to erroneous action are still poorly understood. One possibility relies on the modulation of attentional systems after an error. To explore this possibility, we have designed a Stroop-like visuo-motor task in monkeys that favors incorrect action. Using this task, we previously found that single neurons recorded from the anterior cingulate cortex (ACC) were closely tuned to behavioral performance and, more particularly, that the activity of most neurons was biased towards the evaluation of erroneous action. Here we describe single neurons engaged in both error detection and response alertness processing, whose activation is closely associated with the improvement of subsequent behavioral performance. Specifically, we show that the effect of a warning stimulus on neuronal firing is enhanced after an erroneous response rather than a successful one and that this outcome is correlated with an error rate decrease. Our results suggest that the anterior cingulate cortex, which exhibits this activity, serves as a powerful computational locus for rapid behavioral adaptation.     

Hypermutation and stress adaptation in bacteria

Hypermutability is a phenotype characterized by a moderate to high elevation of spontaneous mutation rates and could result from DNA replication errors, defects in error correction mechanisms and many other causes. The elevated mutation rates are helpful to organisms to adapt to sudden and unforeseen threats to survival. At the same time hypermutability also leads to the generation of many deleterious mutations which offset its adaptive value and therefore disadvantageous. Nevertheless, it is very common in nature, especially among clinical isolates of pathogens. Hypermutability is inherited by indirect (second order) selection along with the beneficial mutations generated. At large population sizes and high mutation rates many cells in the population could concurrently acquire beneficial mutations of varying adaptive (fitness) values. These lineages compete with the ancestral cells and also among themselves for fixation. The one with the ‘fittest’ mutation gets fixed ultimately while the others are lost. This has been called ‘clonal interference’ which puts a speed limit on adaptation. The original clonal interference hypothesis has been modified recently. Nonheritable (transient) hypermtability conferring significant adaptive benefits also occur during stress response although its molecular basis remains controversial. The adaptive benefits of heritable hypermutability are discussed with emphasis on host–pathogen interactions.     

Complex estimation of adaptation abilities of the organism in children using the indices of responsiveness of the cardiovascular system and characteristics of EEG

Based on the complex analysis of the data of cardiointervalographic (CIG) and rheoencephalographic (REG) examination of 37 healthy children and 63 children suffering from bronchial asthma (BA), we identified the most informative indices characterizing reactions of the cardiovascular system under conditions of an active orthostatic test (coefficient of autonomic responsiveness and index of responsiveness of the vessels). We conclude that the relation between the values of these indices allows one to identify the level of strain of the regulatory mechanisms and the state of the adaptation systems of the organism, which determine the adequacy of control of the autonomic sphere. The following gradations can be classified: an optimum level, compensated adaptation disorders (strain and overstrain of the regulatory mechanisms), and decompensated adaptation disorders (exhaustion of the regulatory mechanisms and failure of adaptation). Among clinically healthy children, we found a risk group (about 30%) with manifestations of lowering of the organism’s adaptive abilities. It is shown that groups of healthy children and children suffering from BA differ from each other in the shares of different patterns of EEG and variants of EEG responses to a hyperventilation test. Qualitative and quantitative characteristics of EEG in children, despite high interindividual variability, clearly correlate with the state of adaptation processes. The expedience of a complex approach in estimating the adaptation ability of the child based on the data of CIG, REG and EEG examinations is discussed. Neirofiziologiya/Neurophysiology, Vol. 38, No. 1, pp. 72–84, January–February, 2006.     

Analysis of short- and long-term effects of adaptation in human postural control

 The short-term (i.e., days) and long-term (i.e., months) effects of adaptation to posturography examinations were investigated in 12 normal subjects who were repeatedly examined for five consecutive days and again after 90 days. The examinations were conducted both with eyes open and closed, and the perturbations were evoked by a pseudorandomly applied vibration stimulation to the calf muscles. The evoked anteroposterior responses were analyzed with a method considering adaptation in the slow changes in posture and in the stimulus–response relationship. Repetition of examinations on a daily basis revealed a gradual improvement of postural-control performance. The body sway induced by the stimulation was significantly reduced and the dynamical properties changed. Most of the improvements remained after 90 days, but some parameters such as the complexity of the control system used were increased to the initial level. The results confirm previous observations that postural control contains several partially independent adaptive processes, observed in terms of alteration of posture and as a progressive reduction of body sway induced by stimulation. The method used for the adaptation analysis in this study could be applied to analyze biological systems with multiple individual adaptive processes with different time courses or characteristics, or where the adaptation processes are related to multiple internal or external factors. Received: 2 January 2001 / Accepted in revised form: 27 November 2001     

Seeing your error alters my pointing: observing systematic pointing errors induces sensori-motor after-effects

During the procedure of prism adaptation, subjects execute pointing movements to visual targets under a lateral optical displacement: as consequence of the discrepancy between visual and proprioceptive inputs, their visuo-motor activity is characterized by pointing errors. The perception of such final errors triggers error-correction processes that eventually result into sensori-motor compensation, opposite to the prismatic displacement (i.e., after-effects). Here we tested whether the mere observation of erroneous pointing movements, similar to those executed during prism adaptation, is sufficient to produce adaptation-like after-effects. Neurotypical participants observed, from a first-person perspective, the examiner's arm making incorrect pointing movements that systematically overshot visual targets location to the right, thus simulating a rightward optical deviation. Three classical after-effect measures (proprioceptive, visual and visual-proprioceptive shift) were recorded before and after first-person's perspective observation of pointing errors. Results showed that mere visual exposure to an arm that systematically points on the right-side of a target (i.e., without error correction) produces a leftward after-effect, which mostly affects the observer's proprioceptive estimation of her body midline. In addition, being exposed to such a constant visual error induced in the observer the illusion "to feel" the seen movement. These findings indicate that it is possible to elicit sensori-motor after-effects by mere observation of movement errors.     

Evolutionary dynamics of molecular markers during local adaptation: a case study in Drosophila subobscura

Here we present a correction to our article "Evolutionary dynamics of molecular markers during local adaptation: a case study in Drosophila subobscura ". We have recently detected an error concerning the application of the Ln RH formula – a test to detect positive selection – to our microsatellite data. Here we provide the corrected data and discuss its implications for our overall findings. The corrections presented here have produced some changes relative to our previous results, namely in a locus (dsub14) that presents indications of being affected by positive selection. In general, our populations present less consistent indications of positive selection for this particular locus in both periods studied – between generations 3 and 14 and between generation 14 and 40 of laboratory adaptation. Despite this, the main findings of our study regarding the possibility of positive selection acting on that particular microsatellite still hold. As previously concluded in our article, further studies should be performed on this specific microsatellite locus (and neighboring areas) to elucidate in greater detail the evolutionary forces acting on this specific region of the O chromosome of Drosophila subobscura.     

Law of the Minimum Paradoxes

The “Law of the Minimum” states that growth is controlled by the scarcest resource (limiting factor). This concept was originally applied to plant or crop growth (Justus von Liebig, 1840, Salisbury, Plant physiology, 4th edn., Wadsworth, Belmont, 1992) and quantitatively supported by many experiments. Some generalizations based on more complicated “dose-response” curves were proposed. Violations of this law in natural and experimental ecosystems were also reported. We study models of adaptation in ensembles of similar organisms under load of environmental factors and prove that violation of Liebig’s law follows from adaptation effects. If the fitness of an organism in a fixed environment satisfies the Law of the Minimum then adaptation equalizes the pressure of essential factors and, therefore, acts against the Liebig’s law. This is the the Law of the Minimum paradox: if for a randomly chosen pair “organism–environment” the Law of the Minimum typically holds, then in a well-adapted system, we have to expect violations of this law.     

Detecting multimodality in saccadic reaction time distributions in gap and overlap tasks

In many cases the distribution of saccadic reaction times (SRT) deviates considerably from a unimodal distribution and may often exhibit several peaks. We present a statistical approach to determining the number and form of the individual peaks. The overall density of the reaction times f _i (t), i=1…M obtained in M different experiments with the same subject is described as the sum of K basis functions x _k (t),k=1…K with different weights and an error term. A change in the experimental conditions is assumed to cause a change in the weights, not in the basis functions. We minimize the square of the difference (measured data minus approximation), divided by the error of the data. Incrementing K step by step we determine the necessary number of basis functions. This method is applied to data of six subjects tested in different saccade tasks. We detect five different modes: two in the range 80–140 ms (express modes), two in the range 145–190 ms (fast-regular mode) and one at about 230 ms (slow-regular mode). These modes are located at about the same positions for different subjects. The method presented here not only proves statistically the existence of several modes in SRT distributions but also allows the distributions to be described by a few characteristic numbers that go beyond the mean values and standard deviations. Received: 24 March 1997 / Accepted: 5 December 1997     

Immunolocalization of tight junction proteins in blood vessels in human germinal matrix and cortex

Brain development occurs in a specialized environment maintained by a blood–brain barrier (BBB). An important structural element of the BBB is the endothelial tight junction (TJ). TJs are present during the embryonic period, but BBB impermeability accrues over an extended gestational interval. In studies of human premature infants, we used immunomicroscopy to determine if amounts of the TJ proteins ZO-1, claudin and occludin increase with gestational age in vessels of germinal matrix (GM) and cortex. By 24 weeks postconception (PC), TJ proteins were present in both GM and cortical vessels, but immunoreactivity in the GM of the youngest subjects was less than in older subjects. At 24 weeks PC, TJ protein immunoreactivity in GM vessels was less than in cortical vessels suggesting that TJ maturation progresses along a superficial to deep brain axis. This concept correlates with conclusions from previous analyses of the expression of brain endothelial cell alkaline phosphatase (AP) activity. AP appears in cortical vessels before appearing in deep white matter and GM vessels. Together, these data indicate that differentiation of some functional specializations is still in progress in GM vessels during the third trimester. This maturation could relate to the pathogenesis of germinal matrix hemorrhage–intraventricular hemorrhage.     

A population level computational model of the basal ganglia that generates parkinsonian local field potential activity

Recordings from the basal ganglia’s subthalamic nucleus are acquired via microelectrodes immediately prior to the application of Deep Brain Stimulation (DBS) treatment for Parkinson’s Disease (PD) to assist in the selection of the final point for the implantation of the DBS electrode. The acquired recordings reveal a persistent characteristic beta band peak in the power spectral density function of the Local Field Potential (LFP) signals. This peak is considered to lie at the core of the causality–effect relationships of the parkinsonian pathophysiology. Based on LFPs acquired from human subjects during DBS for PD, we constructed a computational model of the basal ganglia on the population level that generates LFPs to identify the critical pathophysiological alterations that lead to the expression of the beta band peak. To this end, we used experimental data reporting that the strengths of the synaptic connections are modified under dopamine depletion. The hypothesis that the altered dopaminergic modulation may affect both the amplitude and the time course of the postsynaptic potentials is validated by the model. The results suggest a pivotal role of both of these parameters to the pathophysiology of PD.     

Some Aspects of Neuropsychology of Temperament

We studied peculiarities of the autonomic reactions related to emotional experiences in persons with different characteristics of their individuality. To model emotional states, tested subjects were proposed to mentally reproduce situations evoking a sense of joy (positive emotions) and a feeling of grief (negative emotions). During such emotional tests, cardiointervalographic (CIG) indices of the subjects were examined. The following properties of the individuality were taken into account: (i) extraversion/introversion as the temperament parameter, (ii) externality/internality as a characteristic of the locus of psychological self-control, and (iii) extrapunitivity/intropunitivity as a characterological feature manifested in frustration situations. Sympathico-parasympathic influences were more powerful than central influences in regulation of the cardiorhythm in tested subjects with clearly “externally directed” reactions to emotiogenic factors (extraverts, externals, and persons with an extrapunitive type of reaction). Vice versa, shifts of the autonomic balance toward activation of the sympathoadrenal link, relatively low efficacy of baroreflex regulation, and strain of regulatory systems of the organism were observed at a high level of introversion in examined persons. Physiological adaptation to the action of the various stressors, including the emotional ones, is realized mostly by the system cerebral cortex – hypothalamus – hypophysis – adrenal cortex. In turn, the release of adrenalin by adrenal glands activates the reticular formation and, via this structure, the hypothalamus and cerebral limbic system. Thus, it can be supposed that extraverts are characterized by higher thresholds of the emotional sensitivity, which correspond to a higher threshold of activation of the reticular formation and more intense inhibitory cortical influences upon subcortical formations. This is why extraverts manifest weaker, in general, activation of the regulatory mechanisms of the cardiovascular system after the influence of extero- and interoceptive stimuli, while in introverts such activation is more intense.     

End-Point Variability Is Not Noise in Saccade Adaptation

When each of many saccades is made to overshoot its target, amplitude gradually decreases in a form of motor learning called saccade adaptation. Overshoot is induced experimentally by a secondary, backwards intrasaccadic target step (ISS) triggered by the primary saccade. Surprisingly, however, no study has compared the effectiveness of different sizes of ISS in driving adaptation by systematically varying ISS amplitude across different sessions. Additionally, very few studies have examined the feasibility of adaptation with relatively small ISSs. In order to best understand saccade adaptation at a fundamental level, we addressed these two points in an experiment using a range of small, fixed ISS values (from 0° to 1° after a 10° primary target step). We found that significant adaptation occurred across subjects with an ISS as small as 0.25°. Interestingly, though only adaptation in response to 0.25° ISSs appeared to be complete (the magnitude of change in saccade amplitude was comparable to size of the ISS), further analysis revealed that a comparable proportion of the ISS was compensated for across conditions. Finally, we found that ISS size alone was sufficient to explain the magnitude of adaptation we observed; additional factors did not significantly improve explanatory power. Overall, our findings suggest that current assumptions regarding the computation of saccadic error may need to be revisited.