Risk-Sensitivity in Bayesian Sensorimotor Integration

Information processing in the nervous system during sensorimotor tasks with inherent uncertainty has been shown to be consistent with Bayesian integration. Bayes optimal decision-makers are, however, risk-neutral in the sense that they weigh all possibilities based on prior expectation and sensory evidence when they choose the action with highest expected value. In contrast, risk-sensitive decision-makers are sensitive to model uncertainty and bias their decision-making processes when they do inference over unobserved variables. In particular, they allow deviations from their probabilistic model in cases where this model makes imprecise predictions. Here we test for risk-sensitivity in a sensorimotor integration task where subjects exhibit Bayesian information integration when they infer the position of a target from noisy sensory feedback. When introducing a cost associated with subjects'' response, we found that subjects exhibited a characteristic bias towards low cost responses when their uncertainty was high. This result is in accordance with risk-sensitive decision-making processes that allow for deviations from Bayes optimal decision-making in the face of uncertainty. Our results suggest that both Bayesian integration and risk-sensitivity are important factors to understand sensorimotor integration in a quantitative fashion.     

Interference and Shaping in Sensorimotor Adaptations with Rewards

When a perturbation is applied in a sensorimotor transformation task, subjects can adapt and maintain performance by either relying on sensory feedback, or, in the absence of such feedback, on information provided by rewards. For example, in a classical rotation task where movement endpoints must be rotated to reach a fixed target, human subjects can successfully adapt their reaching movements solely on the basis of binary rewards, although this proves much more difficult than with visual feedback. Here, we investigate such a reward-driven sensorimotor adaptation process in a minimal computational model of the task. The key assumption of the model is that synaptic plasticity is gated by the reward. We study how the learning dynamics depend on the target size, the movement variability, the rotation angle and the number of targets. We show that when the movement is perturbed for multiple targets, the adaptation process for the different targets can interfere destructively or constructively depending on the similarities between the sensory stimuli (the targets) and the overlap in their neuronal representations. Destructive interferences can result in a drastic slowdown of the adaptation. As a result of interference, the time to adapt varies non-linearly with the number of targets. Our analysis shows that these interferences are weaker if the reward varies smoothly with the subject''s performance instead of being binary. We demonstrate how shaping the reward or shaping the task can accelerate the adaptation dramatically by reducing the destructive interferences. We argue that experimentally investigating the dynamics of reward-driven sensorimotor adaptation for more than one sensory stimulus can shed light on the underlying learning rules.     

Sensorimotor Learning Biases Choice Behavior: A Learning Neural Field Model for Decision Making

According to a prominent view of sensorimotor processing in primates, selection and specification of possible actions are not sequential operations. Rather, a decision for an action emerges from competition between different movement plans, which are specified and selected in parallel. For action choices which are based on ambiguous sensory input, the frontoparietal sensorimotor areas are considered part of the common underlying neural substrate for selection and specification of action. These areas have been shown capable of encoding alternative spatial motor goals in parallel during movement planning, and show signatures of competitive value-based selection among these goals. Since the same network is also involved in learning sensorimotor associations, competitive action selection (decision making) should not only be driven by the sensory evidence and expected reward in favor of either action, but also by the subject''s learning history of different sensorimotor associations. Previous computational models of competitive neural decision making used predefined associations between sensory input and corresponding motor output. Such hard-wiring does not allow modeling of how decisions are influenced by sensorimotor learning or by changing reward contingencies. We present a dynamic neural field model which learns arbitrary sensorimotor associations with a reward-driven Hebbian learning algorithm. We show that the model accurately simulates the dynamics of action selection with different reward contingencies, as observed in monkey cortical recordings, and that it correctly predicted the pattern of choice errors in a control experiment. With our adaptive model we demonstrate how network plasticity, which is required for association learning and adaptation to new reward contingencies, can influence choice behavior. The field model provides an integrated and dynamic account for the operations of sensorimotor integration, working memory and action selection required for decision making in ambiguous choice situations.     

An approximate stochastic optimal control framework to simulate nonlinear neuro-musculoskeletal models in the presence of noise

Optimal control simulations have shown that both musculoskeletal dynamics and physiological noise are important determinants of movement. However, due to the limited efficiency of available computational tools, deterministic simulations of movement focus on accurately modelling the musculoskeletal system while neglecting physiological noise, and stochastic simulations account for noise while simplifying the dynamics. We took advantage of recent approaches where stochastic optimal control problems are approximated using deterministic optimal control problems, which can be solved efficiently using direct collocation. We were thus able to extend predictions of stochastic optimal control as a theory of motor coordination to include muscle coordination and movement patterns emerging from non-linear musculoskeletal dynamics. In stochastic optimal control simulations of human standing balance, we demonstrated that the inclusion of muscle dynamics can predict muscle co-contraction as minimal effort strategy that complements sensorimotor feedback control in the presence of sensory noise. In simulations of reaching, we demonstrated that nonlinear multi-segment musculoskeletal dynamics enables complex perturbed and unperturbed reach trajectories under a variety of task conditions to be predicted. In both behaviors, we demonstrated how interactions between task constraint, sensory noise, and the intrinsic properties of muscle influence optimal muscle coordination patterns, including muscle co-contraction, and the resulting movement trajectories. Our approach enables a true minimum effort solution to be identified as task constraints, such as movement accuracy, can be explicitly imposed, rather than being approximated using penalty terms in the cost function. Our approximate stochastic optimal control framework predicts complex features, not captured by previous simulation approaches, providing a generalizable and valuable tool to study how musculoskeletal dynamics and physiological noise may alter neural control of movement in both healthy and pathological movements.     

Dynamic updating of distributed neural representations using forward models

In this paper, we present a continuous attractor network model that we hypothesize will give some suggestion of the mechanisms underlying several neural processes such as velocity tuning to visual stimulus, sensory discrimination, sensorimotor transformations, motor control, motor imagery, and imitation. All of these processes share the fundamental characteristic of having to deal with the dynamic integration of motor and sensory variables in order to achieve accurate sensory prediction and/or discrimination. Such principles have already been described in the literature by other high-level modeling studies (Decety and Sommerville in Trends Cogn Sci 7:527–533, 2003; Oztop et al. in Neural Netw 19(3):254–271, 2006; Wolpert et al. in Philos Trans R Soc 358:593–602, 2003). With respect to these studies, our work is more concerned with biologically plausible neural dynamics at a population level. Indeed, we show that a relatively simple extension of the classical neural field models can endow these networks with additional dynamic properties for updating their internal representation using external commands. Moreover, an analysis of the interactions between our model and external inputs also shows interesting properties, which we argue are relevant for a better understanding of the neural processes of the brain.     

The role of spinal manipulation in addressing disordered sensorimotor integration and altered motor control

This review provides an overview of some of the growing body of research on the effects of spinal manipulation on sensory processing, motor output, functional performance and sensorimotor integration. It describes a body of work using somatosensory evoked potentials (SEPs), transcranial magnetic nerve stimulation, and electromyographic techniques to demonstrate neurophysiological changes following spinal manipulation. This work contributes to the understanding of how an initial episode(s) of back or neck pain may lead to ongoing changes in input from the spine which over time lead to altered sensorimotor integration of input from the spine and limbs.     

Sensory ecology and perceptual allocation: new prospects for neural networks

Sensory ecology provides a conceptual framework for considering how animals ought to design sensory systems to capture meaningful information from their environments. The framework has been particularly successful at describing how one should allocate sensory receptors to maximize performance on a given task. Neural networks, in contrast, have made unique contributions to understanding how 'hidden preferences' can emerge as a by-product of sensory design. The two frameworks comprise complementary techniques for understanding the design and the evolution of sensation. This article reviews empirical literature from multiple modalities and levels of sensory processing, considering vision, audition and touch from the viewpoints of sensory ecology and neuroethology. In the process, it presents modifications of extant neural network algorithms that would allow a more effective integration of these diverse approaches. Together, the reviewed literature suggests important advances that can be made by explicitly formulating neural network models in terms of sensory ecology, by incorporating neural costs into models of perceptual evolution and by exploring how such demands interact with historical forces.     

Spatiotemporal dynamics of cortical sensorimotor integration in behaving mice

Tactile information is actively acquired and processed in the brain through concerted interactions between movement and sensation. Somatosensory input is often the result of self-generated movement during the active touch of objects, and conversely, sensory information is used to refine motor control. There must therefore be important interactions between sensory and motor pathways, which we chose to investigate in the mouse whisker sensorimotor system. Voltage-sensitive dye was applied to the neocortex of mice to directly image the membrane potential dynamics of sensorimotor cortex with subcolumnar spatial resolution and millisecond temporal precision. Single brief whisker deflections evoked highly distributed depolarizing cortical sensory responses, which began in the primary somatosensory barrel cortex and subsequently excited the whisker motor cortex. The spread of sensory information to motor cortex was dynamically regulated by behavior and correlated with the generation of sensory-evoked whisker movement. Sensory processing in motor cortex may therefore contribute significantly to active tactile sensory perception.     

Brain–computer interfaces for dissecting cognitive processes underlying sensorimotor control

Sensorimotor control engages cognitive processes such as prediction, learning, and multisensory integration. Understanding the neural mechanisms underlying these cognitive processes with arm reaching is challenging because we currently record only a fraction of the relevant neurons, the arm has nonlinear dynamics, and multiple modalities of sensory feedback contribute to control. A brain–computer interface (BCI) is a well-defined sensorimotor loop with key simplifying advantages that address each of these challenges, while engaging similar cognitive processes. As a result, BCI is becoming recognized as a powerful tool for basic scientific studies of sensorimotor control. Here, we describe the benefits of BCI for basic scientific inquiries and review recent BCI studies that have uncovered new insights into the neural mechanisms underlying sensorimotor control.     

Evolution of a predictive internal model in an embodied and situated agent

We show how simulated robots evolved for the ability to display a context-dependent periodic behavior can spontaneously develop an internal model and rely on it to fulfill their task when sensory stimulation is temporarily unavailable. The analysis of some of the best evolved agents indicates that their internal model operates by anticipating sensory stimuli. More precisely, it anticipates functional properties of the next sensory state rather than the exact state that sensors will assume. The characteristics of the states that are anticipated and of the sensorimotor rules that determine how the agents react to the experienced states, however, ensure that they produce very similar behaviour during normal and blind phases in which sensory stimulation is available or is self-generated by the agent, respectively. Agents’ internal models also ensure an effective transition during the phases in which agents’ internal dynamics is decoupled and re-coupled with the sensorimotor flow. Our results suggest that internal models might have arisen for behavioral reasons and successively exapted for other cognitive functions. Moreover, the obtained results suggest that self-generated internal states should not necessarily match in detail the corresponding sensory states and might rather encode more abstract and motor-oriented information.     

Visual feedback is not necessary for the learning of novel dynamics

Background

When learning to perform a novel sensorimotor task, humans integrate multi-modal sensory feedback such as vision and proprioception in order to make the appropriate adjustments to successfully complete the task. Sensory feedback is used both during movement to control and correct the current movement, and to update the feed-forward motor command for subsequent movements. Previous work has shown that adaptation to stable dynamics is possible without visual feedback. However, it is not clear to what degree visual information during movement contributes to this learning or whether it is essential to the development of an internal model or impedance controller.

Methodology/Principle Findings

We examined the effects of the removal of visual feedback during movement on the learning of both stable and unstable dynamics in comparison with the case when both vision and proprioception are available. Subjects were able to learn to make smooth movements in both types of novel dynamics after learning with or without visual feedback. By examining the endpoint stiffness and force after learning it could be shown that subjects adapted to both types of dynamics in the same way whether they were provided with visual feedback of their trajectory or not. The main effects of visual feedback were to increase the success rate of movements, slightly straighten the path, and significantly reduce variability near the end of the movement.

Conclusions/Significance

These findings suggest that visual feedback of the hand during movement is not necessary for the adaptation to either stable or unstable novel dynamics. Instead vision appears to be used to fine-tune corrections of hand trajectory at the end of reaching movements.     

Cue integration in categorical tasks: insights from audio-visual speech perception

Previous cue integration studies have examined continuous perceptual dimensions (e.g., size) and have shown that human cue integration is well described by a normative model in which cues are weighted in proportion to their sensory reliability, as estimated from single-cue performance. However, this normative model may not be applicable to categorical perceptual dimensions (e.g., phonemes). In tasks defined over categorical perceptual dimensions, optimal cue weights should depend not only on the sensory variance affecting the perception of each cue but also on the environmental variance inherent in each task-relevant category. Here, we present a computational and experimental investigation of cue integration in a categorical audio-visual (articulatory) speech perception task. Our results show that human performance during audio-visual phonemic labeling is qualitatively consistent with the behavior of a Bayes-optimal observer. Specifically, we show that the participants in our task are sensitive, on a trial-by-trial basis, to the sensory uncertainty associated with the auditory and visual cues, during phonemic categorization. In addition, we show that while sensory uncertainty is a significant factor in determining cue weights, it is not the only one and participants' performance is consistent with an optimal model in which environmental, within category variability also plays a role in determining cue weights. Furthermore, we show that in our task, the sensory variability affecting the visual modality during cue-combination is not well estimated from single-cue performance, but can be estimated from multi-cue performance. The findings and computational principles described here represent a principled first step towards characterizing the mechanisms underlying human cue integration in categorical tasks.     

Sensorimotor Integration in Dyslexic Children under Different Sensory Stimulations

Dyslexic children, besides difficulties in mastering literacy, also show poor postural control that might be related to how sensory cues coming from different sensory channels are integrated into proper motor activity. Therefore, the aim of this study was to examine the relationship between sensory information and body sway, with visual and somatosensory information manipulated independent and concurrently, in dyslexic children. Thirty dyslexic and 30 non-dyslexic children were asked to stand as still as possible inside of a moving room either with eyes closed or open and either lightly touching a moveable surface or not for 60 seconds under five experimental conditions: (1) no vision and no touch; (2) moving room; (3) moving bar; (4) moving room and stationary touch; and (5) stationary room and moving bar. Body sway magnitude and the relationship between room/bar movement and body sway were examined. Results showed that dyslexic children swayed more than non-dyslexic children in all sensory condition. Moreover, in those trials with conflicting vision and touch manipulation, dyslexic children swayed less coherent with the stimulus manipulation compared to non-dyslexic children. Finally, dyslexic children showed higher body sway variability and applied higher force while touching the bar compared to non-dyslexic children. Based upon these results, we can suggest that dyslexic children are able to use visual and somatosensory information to control their posture and use the same underlying neural control processes as non-dyslexic children. However, dyslexic children show poorer performance and more variability while relating visual and somatosensory information and motor action even during a task that does not require an active cognitive and motor involvement. Further, in sensory conflict conditions, dyslexic children showed less coherent and more variable body sway. These results suggest that dyslexic children have difficulties in multisensory integration because they may suffer from integrating sensory cues coming from multiple sources.     

Adaptive dynamic programming as a theory of sensorimotor control

Many characteristics of sensorimotor control can be explained by models based on optimization and optimal control theories. However, most of the previous models assume that the central nervous system has access to the precise knowledge of the sensorimotor system and its interacting environment. This viewpoint is difficult to be justified theoretically and has not been convincingly validated by experiments. To address this problem, this paper presents a new computational mechanism for sensorimotor control from a perspective of adaptive dynamic programming (ADP), which shares some features of reinforcement learning. The ADP-based model for sensorimotor control suggests that a command signal for the human movement is derived directly from the real-time sensory data, without the need to identify the system dynamics. An iterative learning scheme based on the proposed ADP theory is developed, along with rigorous convergence analysis. Interestingly, the computational model as advocated here is able to reproduce the motor learning behavior observed in experiments where a divergent force field or velocity-dependent force field was present. In addition, this modeling strategy provides a clear way to perform stability analysis of the overall system. Hence, we conjecture that human sensorimotor systems use an ADP-type mechanism to control movements and to achieve successful adaptation to uncertainties present in the environment.     

Structure Learning in Bayesian Sensorimotor Integration

Previous studies have shown that sensorimotor processing can often be described by Bayesian learning, in particular the integration of prior and feedback information depending on its degree of reliability. Here we test the hypothesis that the integration process itself can be tuned to the statistical structure of the environment. We exposed human participants to a reaching task in a three-dimensional virtual reality environment where we could displace the visual feedback of their hand position in a two dimensional plane. When introducing statistical structure between the two dimensions of the displacement, we found that over the course of several days participants adapted their feedback integration process in order to exploit this structure for performance improvement. In control experiments we found that this adaptation process critically depended on performance feedback and could not be induced by verbal instructions. Our results suggest that structural learning is an important meta-learning component of Bayesian sensorimotor integration.     

Dynamic Excitatory and Inhibitory Gain Modulation Can Produce Flexible,Robust and Optimal Decision-making

Behavioural and neurophysiological studies in primates have increasingly shown the involvement of urgency signals during the temporal integration of sensory evidence in perceptual decision-making. Neuronal correlates of such signals have been found in the parietal cortex, and in separate studies, demonstrated attention-induced gain modulation of both excitatory and inhibitory neurons. Although previous computational models of decision-making have incorporated gain modulation, their abstract forms do not permit an understanding of the contribution of inhibitory gain modulation. Thus, the effects of co-modulating both excitatory and inhibitory neuronal gains on decision-making dynamics and behavioural performance remain unclear. In this work, we incorporate time-dependent co-modulation of the gains of both excitatory and inhibitory neurons into our previous biologically based decision circuit model. We base our computational study in the context of two classic motion-discrimination tasks performed in animals. Our model shows that by simultaneously increasing the gains of both excitatory and inhibitory neurons, a variety of the observed dynamic neuronal firing activities can be replicated. In particular, the model can exhibit winner-take-all decision-making behaviour with higher firing rates and within a significantly more robust model parameter range. It also exhibits short-tailed reaction time distributions even when operating near a dynamical bifurcation point. The model further shows that neuronal gain modulation can compensate for weaker recurrent excitation in a decision neural circuit, and support decision formation and storage. Higher neuronal gain is also suggested in the more cognitively demanding reaction time than in the fixed delay version of the task. Using the exact temporal delays from the animal experiments, fast recruitment of gain co-modulation is shown to maximize reward rate, with a timescale that is surprisingly near the experimentally fitted value. Our work provides insights into the simultaneous and rapid modulation of excitatory and inhibitory neuronal gains, which enables flexible, robust, and optimal decision-making.     

What you feel is what you see: inverse dynamics estimation underlies the resistive sensation of a delayed cursor

How our central nervous system (CNS) learns and exploits relationships between force and motion is a fundamental issue in computational neuroscience. While several lines of evidence have suggested that the CNS predicts motion states and signals from motor commands for control and perception (forward dynamics), it remains controversial whether it also performs the ‘inverse’ computation, i.e. the estimation of force from motion (inverse dynamics). Here, we show that the resistive sensation we experience while moving a delayed cursor, perceived purely from the change in visual motion, provides evidence of the inverse computation. To clearly specify the computational process underlying the sensation, we systematically varied the visual feedback and examined its effect on the strength of the sensation. In contrast to the prevailing theory that sensory prediction errors modulate our perception, the sensation did not correlate with errors in cursor motion due to the delay. Instead, it correlated with the amount of exposure to the forward acceleration of the cursor. This indicates that the delayed cursor is interpreted as a mechanical load, and the sensation represents its visually implied reaction force. Namely, the CNS automatically computes inverse dynamics, using visually detected motions, to monitor the dynamic forces involved in our actions.     

The neural basis of orienting behavior: a computational approach to the escape turn of the cockroach

Using computer simulation, we demonstrate that the information encoded by the ventral giant interneurons is particularly suited to orienting the escape turn of the adult cockroach with a degree of variation seen experimentally. The type of model we present for sensorimotor integration incorporates a simple comparator of sensory evoked GI activity and is extremely robust with respect to underlying assumptions.     

Sensorimotor control of navigation in arthropod and artificial systems

Arthropods exhibit highly efficient solutions to sensorimotor navigation problems. They thus provide a source of inspiration and ideas to robotics researchers. At the same time, attempting to re-engineer these mechanisms in robot hardware and software provides useful insights into how the natural systems might work. This paper reviews three examples of arthropod sensorimotor control systems that have been implemented and tested on robots. First we discuss visual control mechanisms of flies, such as the optomotor reflex and collision avoidance, that have been replicated in analog VLSI (very large scale integration) hardware and used to produce corrective behavior in robot vehicles. Then, we present a robot model of auditory localization in the cricket; and discuss integration of this behavior with the optomotor behavior previously described. Finally we present a model of olfactory search in the moth, which makes use of several sensory cues, and has also been tested using robot hardware. We discuss some of the similarities and differences of the solutions obtained.     

EEG Reflection of Clinical Effects Dynamics during Botulinum Toxin Therapy of Movement Disorders under Long-Term Consciousness Disorders

When studying the dynamics of brain electric activity (BEA) in the course of botulinum toxin (Incobotulinumtoxin A) injections into all hypertonic muscles of 16 patients suffering from post-coma longterm consciousness disorders, BEA changes were observed to start within the first minutes after the first intramuscular injection of botulinum toxin, while not being associated with any pain stimulus response. Despite the diversity of brain lesions in these patients, the dynamics of BEA reorganisation during botulinum toxin therapy showed the presence of pathological sensorimotor integration involving the whole brain in case of long-term consciousness disorders. In our opinion, it is expressed in a pathological state developing in the brain, which affects the polyfunctional capabilities of neurons. This state is unbalanced due to destabilizing effect at the beginning of the reduction by pathological hyperafferentation from muscles and, in principle, can be disrupted at a complete reduction. It is, however, maintained by memory matrix, which manifests itself in the shape of oscillation process during its disruption when the newly build-up BEA is occasionally displaced by the previous one. During the reduction of sensory hyperafferentation, BEA reorganisation initially involved the motor cortex (in 15 out of 16 patients) and then spread to other brain areas, including those responsible for the higher mental functions, vision, and hearing. The first transiently decreased muscle tone (in all patients) and improvement of neurological signs of awareness (in some patients) were observed immediately after injecting all hypertonic muscles with widespread BEA changes. The appearance of high-frequency EEG activity in the frontal areas was considered as a universal marker of an improved functional state of the brain during the disruption of pathological sensorimotor integration even before clinical signs, suggesting neuronal readiness to maintain various activities. In view of data obtained, the first-stage clinical measures in patients with disorders of consciousness should focus on disrupting pathological sensorimotor integration, opening a way for restoring neuronal functional activity, provided that all “pathological afferents” are reduced.