Optimal isn’t good enough

The notion that biological systems come to embody optimal solutions seems consistent with the competitive drive of evolution. It has been used to interpret many examples of sensorimotor behavior. It is attractive from the viewpoint of control engineers because it solves the redundancy problem by identifying the one optimal motor strategy out of many similarly acceptable possibilities. This perspective examines whether there is sufficient basis to apply the formal engineering tools of optimal control to a reductionist understanding of biological systems. For an experimental biologist, this translates into whether the theory of optimal control generates nontrivial and testable hypotheses that accurately predict novel phenomena, ideally at deeper levels of structure than the observable behavior. The methodology of optimal control is applicable when there is (i) a single, known cost function to be optimized, (ii) an invertible model of the plant, and (iii) simple noise interfering with optimal performance. None of these is likely to be true for biological organisms. Furthermore, their motivation is usually good-enough rather than globally optimal behavior. Even then, the performance of a biological organism is often much farther from optimal than the physical limits of its hardware because the brain is continuously testing the acceptable limits of performance as well as just performing the task. This perspective considers an alternative strategy called ??good-enough?? control, in which the organism uses trial-and-error learning to acquire a repertoire of sensorimotor behaviors that are known to be useful, but not necessarily optimal. This leads to a diversity of solutions that tends to confer robustness on the individual organism and its evolution. It is also more consistent with the capabilities of higher sensorimotor structures, such as cerebral cortex, which seems to be designed to classify and recall complex sets of information, thereby allowing the organism to learn from experience, rather than to compute new strategies online. Optimal control has been a useful metaphor for understanding some superficial aspects of motor psychophysics. Reductionists who want to understand the underlying neural mechanisms need to move on.     

Studying sensorimotor integration in insects

Sensorimotor integration is a field rich in theory backed by a large body of psychophysical evidence. Relating the underlying neural circuitry to these theories has, however, been more challenging. With a wide array of complex behaviors coordinated by their small brains, insects provide powerful model systems to study key features of sensorimotor integration at a mechanistic level. Insect neural circuits perform both hard-wired and learned sensorimotor transformations. They modulate their neural processing based on both internal variables, such as the animal's behavioral state, and external ones, such as the time of day. Here we present some studies using insect model systems that have produced insights, at the level of individual neurons, about sensorimotor integration and the various ways in which it can be modified by context.     

Mechanical aspects of legged locomotion control

We review the mechanical components of an approach to motion science that enlists recent progress in neurophysiology, biomechanics, control systems engineering, and non-linear dynamical systems to explore the integration of muscular, skeletal, and neural mechanics that creates effective locomotor behavior. We use rapid arthropod terrestrial locomotion as the model system because of the wealth of experimental data available. With this foundation, we list a set of hypotheses for the control of movement, outline their mathematical underpinning and show how they have inspired the design of the hexapedal robot, RHex.     

Adaptive dynamic programming as a theory of sensorimotor control

Many characteristics of sensorimotor control can be explained by models based on optimization and optimal control theories. However, most of the previous models assume that the central nervous system has access to the precise knowledge of the sensorimotor system and its interacting environment. This viewpoint is difficult to be justified theoretically and has not been convincingly validated by experiments. To address this problem, this paper presents a new computational mechanism for sensorimotor control from a perspective of adaptive dynamic programming (ADP), which shares some features of reinforcement learning. The ADP-based model for sensorimotor control suggests that a command signal for the human movement is derived directly from the real-time sensory data, without the need to identify the system dynamics. An iterative learning scheme based on the proposed ADP theory is developed, along with rigorous convergence analysis. Interestingly, the computational model as advocated here is able to reproduce the motor learning behavior observed in experiments where a divergent force field or velocity-dependent force field was present. In addition, this modeling strategy provides a clear way to perform stability analysis of the overall system. Hence, we conjecture that human sensorimotor systems use an ADP-type mechanism to control movements and to achieve successful adaptation to uncertainties present in the environment.     

Dynamically repairing and replacing neural networks: using hybrid computational and biological tools

The debilitating effects of injury to the nervous system can have a profound effect on daily life activities of the injured person. In this article, we present a project overview in which we are utilizing computational and biological principles, along with simulation and experimentation, to create a realistic computational model of natural and injured sensorimotor control systems. Through the development of hybrid in silico/biological coadaptive symbiotic systems, the goal is to create new technologies that yield transformative neuroprosthetic rehabilitative solutions and a new test bed for the development of integrative medical devices for the repair and enhancement of biological systems.     

Computational mechanisms of sensorimotor control

In order to generate skilled and efficient actions, the motor system must find solutions to several problems inherent in sensorimotor control, including nonlinearity, nonstationarity, delays, redundancy, uncertainty, and noise. We review these problems and five computational mechanisms that the brain may use to limit their deleterious effects: optimal feedback control, impedance control, predictive control, Bayesian decision theory, and sensorimotor learning. Together, these computational mechanisms allow skilled and fluent sensorimotor behavior.     

Using hardware models to quantify sensory data acquisition across the rat vibrissal array

Our laboratory investigates how animals acquire sensory data to understand the neural computations that permit complex sensorimotor behaviors. We use the rat whisker system as a model to study active tactile sensing; our aim is to quantitatively describe the spatiotemporal structure of incoming sensory information to place constraints on subsequent neural encoding and processing. In the first part of this paper we describe the steps in the development of a hardware model (a 'sensobot') of the rat whisker array that can perform object feature extraction. We show how this model provides insights into the neurophysiology and behavior of the real animal. In the second part of this paper, we suggest that sensory data acquisition across the whisker array can be quantified using the complete derivative. We use the example of wall-following behavior to illustrate that computing the appropriate spatial gradients across a sensor array would enable an animal or mobile robot to predict the sensory data that will be acquired at the next time step.     

Brain–computer interfaces for dissecting cognitive processes underlying sensorimotor control

Sensorimotor control engages cognitive processes such as prediction, learning, and multisensory integration. Understanding the neural mechanisms underlying these cognitive processes with arm reaching is challenging because we currently record only a fraction of the relevant neurons, the arm has nonlinear dynamics, and multiple modalities of sensory feedback contribute to control. A brain–computer interface (BCI) is a well-defined sensorimotor loop with key simplifying advantages that address each of these challenges, while engaging similar cognitive processes. As a result, BCI is becoming recognized as a powerful tool for basic scientific studies of sensorimotor control. Here, we describe the benefits of BCI for basic scientific inquiries and review recent BCI studies that have uncovered new insights into the neural mechanisms underlying sensorimotor control.     

Deconstructing memory in Drosophila

Unlike most organ systems, which have evolved to maintain homeostasis, the brain has been selected to sense and adapt to environmental stimuli by constantly altering interactions in a gene network that functions within a larger neural network. This unique feature of the central nervous system provides a remarkable plasticity of behavior, but also makes experimental investigations challenging. Each experimental intervention ramifies through both gene and neural networks, resulting in unpredicted and sometimes confusing phenotypic adaptations. Experimental dissection of mechanisms underlying behavioral plasticity ultimately must accomplish an integration across many levels of biological organization, including genetic pathways acting within individual neurons, neural network interactions which feed back to gene function, and phenotypic observations at the behavioral level. This dissection will be more easily accomplished for model systems such as Drosophila, which, compared with mammals, have relatively simple and manipulable nervous systems and genomes. The evolutionary conservation of behavioral phenotype and the underlying gene function ensures that much of what we learn in such model systems will be relevant to human cognition. In this essay, we have not attempted to review the entire Drosophila memory field. Instead, we have tried to discuss particular findings that provide some level of intellectual synthesis across three levels of biological organization: behavior, neural circuitry and biochemical pathways. We have attempted to use this integrative approach to evaluate distinct mechanistic hypotheses, and to propose critical experiments that will advance this field.     

Biological pathways to adaptability--interactions between genome, epigenome, nervous system and environment for adaptive behavior

Because living systems depend on their environment, the evolution of environmental adaptability is inseparable from the evolution of life itself (Pross 2003). In animals and humans, environmental adaptability extends further to adaptive behavior. It has recently emerged that individual adaptability depends on the interaction of adaptation mechanisms at diverse functional levels. This interaction enables the integration of genetic, epigenetic and environmental factors for coordinated regulation of adaptations. In this review, we first present the basis for the regulation of adaptation mechanisms across functional levels. We then focus on neuronal activity-regulated adaptation mechanisms that involve the regulation of genes, noncoding DNA (ncDNA), ncRNAs and proteins to change the structural and functional properties of neurons. Finally, we discuss a selection of these important neuronal activity-regulated molecules and their effects on brain structure and function and on behavior. Most of the evidence so far is based on sampling of animal tissue or post-mortem studies in humans. However, we also present techniques that combine genetic with behavioral and neurophysiological measures in humans (e.g. genetic imaging) and discuss their potential and limitations. We argue that we need to understand how neuronal activity-dependent adaptation mechanisms integrate genetic, epigenetic and experience-dependent signals in order to explain individual variations in behavior and cognitive performance.     

Deficient Grip Force Control in Schizophrenia: Behavioral and Modeling Evidence for Altered Motor Inhibition and Motor Noise

Whether upper limb sensorimotor control is affected in schizophrenia and how underlying pathological mechanisms may potentially intervene in these deficits is still being debated. We tested voluntary force control in schizophrenia patients and used a computational model in order to elucidate potential cerebral mechanisms underlying sensorimotor deficits in schizophrenia. A visuomotor grip force-tracking task was performed by 17 medicated and 6 non-medicated patients with schizophrenia (DSM-IV) and by 15 healthy controls. Target forces in the ramp-hold-and-release paradigm were set to 5N and to 10% maximal voluntary grip force. Force trajectory was analyzed by performance measures and Principal Component Analysis (PCA). A computational model incorporating neural control signals was used to replicate the empirically observed motor behavior and to explore underlying neural mechanisms. Grip task performance was significantly lower in medicated and non-medicated schizophrenia patients compared to controls. Three behavioral variables were significantly higher in both patient groups: tracking error (by 50%), coefficient of variation of force (by 57%) and duration of force release (up by 37%). Behavioral performance did not differ between patient groups. Computational simulation successfully replicated these findings and predicted that decreased motor inhibition, together with an increased signal-dependent motor noise, are sufficient to explain the observed motor deficits in patients. PCA also suggested altered motor inhibition as a key factor differentiating patients from control subjects: the principal component representing inhibition correlated with clinical severity. These findings show that schizophrenia affects voluntary sensorimotor control of the hand independent of medication, and suggest that reduced motor inhibition and increased signal-dependent motor noise likely reflect key pathological mechanisms of the sensorimotor deficit.     

Sensorimotor Learning Biases Choice Behavior: A Learning Neural Field Model for Decision Making

According to a prominent view of sensorimotor processing in primates, selection and specification of possible actions are not sequential operations. Rather, a decision for an action emerges from competition between different movement plans, which are specified and selected in parallel. For action choices which are based on ambiguous sensory input, the frontoparietal sensorimotor areas are considered part of the common underlying neural substrate for selection and specification of action. These areas have been shown capable of encoding alternative spatial motor goals in parallel during movement planning, and show signatures of competitive value-based selection among these goals. Since the same network is also involved in learning sensorimotor associations, competitive action selection (decision making) should not only be driven by the sensory evidence and expected reward in favor of either action, but also by the subject''s learning history of different sensorimotor associations. Previous computational models of competitive neural decision making used predefined associations between sensory input and corresponding motor output. Such hard-wiring does not allow modeling of how decisions are influenced by sensorimotor learning or by changing reward contingencies. We present a dynamic neural field model which learns arbitrary sensorimotor associations with a reward-driven Hebbian learning algorithm. We show that the model accurately simulates the dynamics of action selection with different reward contingencies, as observed in monkey cortical recordings, and that it correctly predicted the pattern of choice errors in a control experiment. With our adaptive model we demonstrate how network plasticity, which is required for association learning and adaptation to new reward contingencies, can influence choice behavior. The field model provides an integrated and dynamic account for the operations of sensorimotor integration, working memory and action selection required for decision making in ambiguous choice situations.     

Information-Driven Active Audio-Visual Source Localization

We present a system for sensorimotor audio-visual source localization on a mobile robot. We utilize a particle filter for the combination of audio-visual information and for the temporal integration of consecutive measurements. Although the system only measures the current direction of the source, the position of the source can be estimated because the robot is able to move and can therefore obtain measurements from different directions. These actions by the robot successively reduce uncertainty about the source’s position. An information gain mechanism is used for selecting the most informative actions in order to minimize the number of actions required to achieve accurate and precise position estimates in azimuth and distance. We show that this mechanism is an efficient solution to the action selection problem for source localization, and that it is able to produce precise position estimates despite simplified unisensory preprocessing. Because of the robot’s mobility, this approach is suitable for use in complex and cluttered environments. We present qualitative and quantitative results of the system’s performance and discuss possible areas of application.     

Emergence of functional hierarchy in a multiple timescale neural network model: a humanoid robot experiment

It is generally thought that skilled behavior in human beings results from a functional hierarchy of the motor control system, within which reusable motor primitives are flexibly integrated into various sensori-motor sequence patterns. The underlying neural mechanisms governing the way in which continuous sensori-motor flows are segmented into primitives and the way in which series of primitives are integrated into various behavior sequences have, however, not yet been clarified. In earlier studies, this functional hierarchy has been realized through the use of explicit hierarchical structure, with local modules representing motor primitives in the lower level and a higher module representing sequences of primitives switched via additional mechanisms such as gate-selecting. When sequences contain similarities and overlap, however, a conflict arises in such earlier models between generalization and segmentation, induced by this separated modular structure. To address this issue, we propose a different type of neural network model. The current model neither makes use of separate local modules to represent primitives nor introduces explicit hierarchical structure. Rather than forcing architectural hierarchy onto the system, functional hierarchy emerges through a form of self-organization that is based on two distinct types of neurons, each with different time properties ("multiple timescales"). Through the introduction of multiple timescales, continuous sequences of behavior are segmented into reusable primitives, and the primitives, in turn, are flexibly integrated into novel sequences. In experiments, the proposed network model, coordinating the physical body of a humanoid robot through high-dimensional sensori-motor control, also successfully situated itself within a physical environment. Our results suggest that it is not only the spatial connections between neurons but also the timescales of neural activity that act as important mechanisms leading to functional hierarchy in neural systems.     

I See What You Mean: How Attentional Selection Is Shaped by Ascribing Intentions to Others

The ability to understand and predict others’ behavior is essential for successful interactions. When making predictions about what other humans will do, we treat them as intentional systems and adopt the intentional stance, i.e., refer to their mental states such as desires and intentions. In the present experiments, we investigated whether the mere belief that the observed agent is an intentional system influences basic social attention mechanisms. We presented pictures of a human and a robot face in a gaze cuing paradigm and manipulated the likelihood of adopting the intentional stance by instruction: in some conditions, participants were told that they were observing a human or a robot, in others, that they were observing a human-like mannequin or a robot whose eyes were controlled by a human. In conditions in which participants were made to believe they were observing human behavior (intentional stance likely) gaze cuing effects were significantly larger as compared to conditions when adopting the intentional stance was less likely. This effect was independent of whether a human or a robot face was presented. Therefore, we conclude that adopting the intentional stance when observing others’ behavior fundamentally influences basic mechanisms of social attention. The present results provide striking evidence that high-level cognitive processes, such as beliefs, modulate bottom-up mechanisms of attentional selection in a top-down manner.     

Visual Feedback Dominates the Sense of Agency for Brain-Machine Actions

Recent advances in neuroscience and engineering have led to the development of technologies that permit the control of external devices through real-time decoding of brain activity (brain-machine interfaces; BMI). Though the feeling of controlling bodily movements (sense of agency; SOA) has been well studied and a number of well-defined sensorimotor and cognitive mechanisms have been put forth, very little is known about the SOA for BMI-actions. Using an on-line BMI, and verifying that our subjects achieved a reasonable level of control, we sought to describe the SOA for BMI-mediated actions. Our results demonstrate that discrepancies between decoded neural activity and its resultant real-time sensory feedback are associated with a decrease in the SOA, similar to SOA mechanisms proposed for bodily actions. However, if the feedback discrepancy serves to correct a poorly controlled BMI-action, then the SOA can be high and can increase with increasing discrepancy, demonstrating the dominance of visual feedback on the SOA. Taken together, our results suggest that bodily and BMI-actions rely on common mechanisms of sensorimotor integration for agency judgments, but that visual feedback dominates the SOA in the absence of overt bodily movements or proprioceptive feedback, however erroneous the visual feedback may be.     

Some emerging principles underlying the physical properties, biological actions, and utility of vitrification solutions

Vitrification solutions are aqueous cryoprotectant solutions which do not freeze when cooled at moderate rates to very low temperatures. Vitrification solutions have been used with great success for the cryopreservation of some biological systems but have been less successful or unsuccessful with other systems, and more fundamental knowledge about vitrification solutions is required. The purpose of the present survey is to show that a general understanding of the physical behavior and biological effects of vitrification solutions, as well as an understanding of the conditions under which vitrification solutions are required, is gradually emerging. Detailed nonequilibrium phase diagram information in combination with specific information on the tolerance of biological systems to ice and to cryoprotectant at subzero temperatures provides a quantitative theoretical basis for choosing between vitrification and freezing. The vitrification behavior of mixtures of cryoprotective agents during cooling is predictable from the behavior of the individual agents, and the behavior of individual agents is gradually becoming predictable from the details of their molecular structures. Progress is continuing concerning the elucidation of mechanisms and cellular sites of toxicity and mechanisms for the reduction of toxicity. Finally, important new information is rapidly emerging concerning the crystallization of previously vitrified cryoprotectant solutions during warming. It appears that vitrification tendency, toxicity, and devitrification all depend on subtle variations in the organization of water around dissolved substances.     

Visual motion,binocular correspondence and binocular rivalry

The human pupillary control system has been the subject of interest to biologists and engineers as an example of a sensorimotor reflex which can be embedded in a control system paradigm. We present a nonlinear feedback model whose compact structure allows us to hypothesize possible physiological mechanisms which generate the proper behavior of the pupil system. The important pupil responses, including pupil size effect, asymmetry, and response to high-frequency stimuli, are defined. This model was simulated on a digital computer and comparisons to the paradigm experimental responses were performed, demonstrating a fit to each of the observed conditions. Improvements on previous models are discussed.