Inferring visuomotor priors for sensorimotor learning

Sensorimotor learning has been shown to depend on both prior expectations and sensory evidence in a way that is consistent with Bayesian integration. Thus, prior beliefs play a key role during the learning process, especially when only ambiguous sensory information is available. Here we develop a novel technique to estimate the covariance structure of the prior over visuomotor transformations--the mapping between actual and visual location of the hand--during a learning task. Subjects performed reaching movements under multiple visuomotor transformations in which they received visual feedback of their hand position only at the end of the movement. After experiencing a particular transformation for one reach, subjects have insufficient information to determine the exact transformation, and so their second reach reflects a combination of their prior over visuomotor transformations and the sensory evidence from the first reach. We developed a Bayesian observer model in order to infer the covariance structure of the subjects' prior, which was found to give high probability to parameter settings consistent with visuomotor rotations. Therefore, although the set of visuomotor transformations experienced had little structure, the subjects had a strong tendency to interpret ambiguous sensory evidence as arising from rotation-like transformations. We then exposed the same subjects to a highly-structured set of visuomotor transformations, designed to be very different from the set of visuomotor rotations. During this exposure the prior was found to have changed significantly to have a covariance structure that no longer favored rotation-like transformations. In summary, we have developed a technique which can estimate the full covariance structure of a prior in a sensorimotor task and have shown that the prior over visuomotor transformations favor a rotation-like structure. Moreover, through experience of a novel task structure, participants can appropriately alter the covariance structure of their prior.     

Extensive cochleotopic mapping of human auditory cortical fields obtained with phase-encoding FMRI

The primary sensory cortices are characterized by a topographical mapping of basic sensory features which is considered to deteriorate in higher-order areas in favor of complex sensory features. Recently, however, retinotopic maps were also discovered in the higher-order visual, parietal and prefrontal cortices. The discovery of these maps enabled the distinction between visual regions, clarified their function and hierarchical processing. Could such extension of topographical mapping to high-order processing regions apply to the auditory modality as well? This question has been studied previously in animal models but only sporadically in humans, whose anatomical and functional organization may differ from that of animals (e.g. unique verbal functions and Heschl''s gyrus curvature). Here we applied fMRI spectral analysis to investigate the cochleotopic organization of the human cerebral cortex. We found multiple mirror-symmetric novel cochleotopic maps covering most of the core and high-order human auditory cortex, including regions considered non-cochleotopic, stretching all the way to the superior temporal sulcus. These maps suggest that topographical mapping persists well beyond the auditory core and belt, and that the mirror-symmetry of topographical preferences may be a fundamental principle across sensory modalities.     

Sensory signals during active versus passive movement

Our sensory systems are simultaneously activated as the result of our own actions and changes in the external world. The ability to distinguish self-generated sensory events from those that arise externally is thus essential for perceptual stability and accurate motor control. Recently, progress has been made towards understanding how this distinction is made. It has been proposed that an internal prediction of the consequences of our actions is compared to the actual sensory input to cancel the resultant self-generated activation. Evidence in support of this hypothesis has been obtained for early stages of sensory processing in the vestibular, visual and somatosensory systems. These findings have implications for the sensory-motor transformations that are needed to guide behavior.     

Correction of Distortion in Flattened Representations of the Cortical Surface Allows Prediction of V1-V3 Functional Organization from Anatomy

Several domains of neuroscience offer map-like models that link location on the cortical surface to properties of sensory representation. Within cortical visual areas V1, V2, and V3, algebraic transformations can relate position in the visual field to the retinotopic representation on the flattened cortical sheet. A limit to the practical application of this structure-function model is that the cortex, while topologically a two-dimensional surface, is curved. Flattening of the curved surface to a plane unavoidably introduces local geometric distortions that are not accounted for in idealized models. Here, we show that this limitation is overcome by correcting the geometric distortion induced by cortical flattening. We use a mass-spring-damper simulation to create a registration between functional MRI retinotopic mapping data of visual areas V1, V2, and V3 and an algebraic model of retinotopy. This registration is then applied to the flattened cortical surface anatomy to create an anatomical template that is linked to the algebraic retinotopic model. This registered cortical template can be used to accurately predict the location and retinotopic organization of these early visual areas from cortical anatomy alone. Moreover, we show that prediction accuracy remains when extrapolating beyond the range of data used to inform the model, indicating that the registration reflects the retinotopic organization of visual cortex. We provide code for the mass-spring-damper technique, which has general utility for the registration of cortical structure and function beyond the visual cortex.     

Visual perception of rigidity of solid shape

Summary Organisms react to objective properties of bodies in their visual field instead of to the perpetually changing retinal images of those bodies. We show how such a faculty can be mechanized. The organism synthesizes an internal model of the external object, that is a bundle of expectations of how the visual input will transform in response to the organism's exploratory movements. We deduce the necessary structure of the internal model and we show how the organism can extract this structure from the invariant features of the sensory input transformations. With this internal model it is possible to predict the subsequent aspects (contours) of the visual object as the spatial relations of organism and object change.     

Visuomotor transformations: early cortical mechanisms of reaching

Recent studies of visually guided reaching in monkeys support the hypothesis that the visuomotor transformations underlying arm movements to spatial targets involve a parallel mechanism that simultaneously engages functionally related frontal and parietal areas linked by reciprocal cortico-cortical connections. The neurons in these areas possess similar combinations of response properties. The multimodal combinatorial properties of these neurons and the gradient architecture of the parieto-frontal network emerge as a potential substrate to link the different sensory and motor signals that arise during reaching behavior into common hybrid reference frames. This convergent combinatorial process is evident at early stages of visual information processing in the occipito-parietal cortex, suggesting the existence of re-entrant motor influences on cortical areas once believed to have only visual functions.     

Dynamic Electrode-to-Image (DETI) mapping reveals the human brain’s spatiotemporal code of visual information

A number of neuroimaging techniques have been employed to understand how visual information is transformed along the visual pathway. Although each technique has spatial and temporal limitations, they can each provide important insights into the visual code. While the BOLD signal of fMRI can be quite informative, the visual code is not static and this can be obscured by fMRI’s poor temporal resolution. In this study, we leveraged the high temporal resolution of EEG to develop an encoding technique based on the distribution of responses generated by a population of real-world scenes. This approach maps neural signals to each pixel within a given image and reveals location-specific transformations of the visual code, providing a spatiotemporal signature for the image at each electrode. Our analyses of the mapping results revealed that scenes undergo a series of nonuniform transformations that prioritize different spatial frequencies at different regions of scenes over time. This mapping technique offers a potential avenue for future studies to explore how dynamic feedforward and recurrent processes inform and refine high-level representations of our visual world.     

Migraine aura dynamics after reverse retinotopic mapping of weak excitation waves in the primary visual cortex

 A kinematical model for excitable wave propagation is analyzed to describe the dynamics of a typical neurological symptom of migraine. The kinematical model equation is solved analytically for a linear dependency between front curvature and velocity. The resulting wave starts from an initial excitation and moves in the medium that represents the primary visual cortex. Due to very weak excitability the wave propagates only across a confined area and eventually disappears. This cortical excitation pattern is projected onto a visual hemifield by reverse retinotopic mapping. Weak excitability explains the confined appearance of aura symptoms in time and sensory space. The affected area in the visual field matches in growth and form the one reported by migraine sufferers. The results can be extended from visual to tactile and to other sensory symptoms. If the spatiotemporal pattern from our model can be matched in future investigations with those from introspectives, it would allow one to draw conclusions on topographic mapping of sensory input in human cortex. Received: 25 April 2002 / Accepted: 20 February 2003 / Published online: 20 May 2003 RID="*" ID="*" Present address: M. A. Dahlem Leibniz-Institut für Neurobiologie, Brenneckestr. 6, 39118 Magdeburg, Germany Acknowledgements. We would like to thank V. Zykov for useful discussions on wave Propagation, and one of us (MAD) would like to thank Ed Chronicle for useful discussions on functional excitability. This project was supported by a scholarship Landesstipendium Sachsen-Anhalt to MAD. Correspondence to: M. A. Dahlem (e-mail: dahlem@ifn-magdeburg.de)     

听觉皮层信号处理

听觉系统和视觉系统的不同之处在于：听觉系统在外周感受器和听皮层间具有更长的皮层下通路和更多的突触联系。该特殊结构反应了听觉系统从复杂听觉环境中提取与行为相关信号的机制与其他感觉系统不同。听皮层神经信号处理包括两种重要的转换机制,声音信号的非同构转换以及从声音感受到知觉层面的转换。听觉皮层神经编码机制同时也受到听觉反馈和语言或发声过程中发声信号的调控。听觉神经科学家和生物医学工程师所面临的挑战便是如何去理解大脑中这些转换的编码机制。我将会用我实验室最近的一些发现来阐述听觉信号是如何在原听皮层中进行处理的,并讨论其对于言语和音乐在大脑中的处理机制以及设计神经替代装置诸如电子耳蜗的意义。我们使用了结合神经电生理技术和量化工程学的方法来研究这些问题。     

Control of retinal growth and axon divergence at the chiasm: lessons from Xenopus

Metamorphosis in frogs is a critical developmental process through which a tadpole changes into an adult froglet. Metamorphic changes include external morphological transformations as well as important changes in the wiring of sensory organs and central nervous system. This review aims to provide an overview on the events that occur in the visual system of metamorphosing amphibians and to discuss recent studies that provide new insight into the molecular mechanisms that control changes in the retinal growth pattern as well as the formation of new axonal pathways in the central nervous system. BioEssays 23:319-326, 2001.     

Rapid Extraction of Lexical Tone Phonology in Chinese Characters: A Visual Mismatch Negativity Study

Background

In alphabetic languages, emerging evidence from behavioral and neuroimaging studies shows the rapid and automatic activation of phonological information in visual word recognition. In the mapping from orthography to phonology, unlike most alphabetic languages in which there is a natural correspondence between the visual and phonological forms, in logographic Chinese, the mapping between visual and phonological forms is rather arbitrary and depends on learning and experience. The issue of whether the phonological information is rapidly and automatically extracted in Chinese characters by the brain has not yet been thoroughly addressed.

Methodology/Principal Findings

We continuously presented Chinese characters differing in orthography and meaning to adult native Mandarin Chinese speakers to construct a constant varying visual stream. In the stream, most stimuli were homophones of Chinese characters: The phonological features embedded in these visual characters were the same, including consonants, vowels and the lexical tone. Occasionally, the rule of phonology was randomly violated by characters whose phonological features differed in the lexical tone.

Conclusions/Significance

We showed that the violation of the lexical tone phonology evoked an early, robust visual response, as revealed by whole-head electrical recordings of the visual mismatch negativity (vMMN), indicating the rapid extraction of phonological information embedded in Chinese characters. Source analysis revealed that the vMMN was involved in neural activations of the visual cortex, suggesting that the visual sensory memory is sensitive to phonological information embedded in visual words at an early processing stage.     

Neuroimaging of cognitive functions in human parietal cortex

Functional neuroimaging has proven highly valuable in mapping human sensory regions, particularly visual areas in occipital cortex. Recent evidence suggests that human parietal cortex may also consist of numerous specialized subregions similar to those reported in neurophysiological studies of non-human primates. However, parietal activation generalizes across a wide variety of cognitive tasks and the extension of human brain mapping into higher-order "association cortex" may prove to be a challenge.     

Predictability of visual perturbation during locomotion: implications for corrective efference copy signaling

In guiding adaptive behavior, efference copy signals or corollary discharge are traditionally considered to serve as predictors of self-generated sensory inputs and by interfering with their central processing are able to counter unwanted consequences of an animal??s own actions. Here, in a speculative reflection on this issue, we consider a different functional role for such intrinsic predictive signaling, namely in stabilizing gaze during locomotion where resultant changes in head orientation in space require online compensatory eye movements in order to prevent retinal image slip. The direct activation of extraocular motoneurons by locomotor-related efference copies offers a prospective substrate for assisting self-motion derived sensory feedback, rather than being subtracted from the sensory signal to eliminate unwanted reafferent information. However, implementing such a feed-forward mechanism would be critically dependent on an appropriate phase coupling between rhythmic propulsive movement and resultant head/visual image displacement. We used video analyzes of actual locomotor behavior and basic theoretical modeling to evaluate head motion during stable locomotion in animals as diverse as Xenopus laevis tadpoles, teleost fish and horses in order to assess the potential suitability of spinal efference copies to the stabilization of gaze during locomotion. In all three species, and therefore regardless of aquatic or terrestrial environment, the head displacements that accompanied locomotor action displayed a strong correlative spatio-temporal relationship in correspondence with a potential predictive value for compensatory eye adjustments. Although spinal central pattern generator-derived efference copies offer appropriately timed commands for extraocular motor control during self-generated motion, it is likely that precise image stabilization requires the additional contributions of sensory feedback signals. Nonetheless, the predictability of the visual consequences of stereotyped locomotion renders intrinsic efference copy signaling an appealing mechanism for offsetting these disturbances, thus questioning the exclusive role traditionally ascribed to sensory-motor transformations in stabilizing gaze during vertebrate locomotion.     

From Sensory Signals to Modality-Independent Conceptual Representations: A Probabilistic Language of Thought Approach

People learn modality-independent, conceptual representations from modality-specific sensory signals. Here, we hypothesize that any system that accomplishes this feat will include three components: a representational language for characterizing modality-independent representations, a set of sensory-specific forward models for mapping from modality-independent representations to sensory signals, and an inference algorithm for inverting forward models—that is, an algorithm for using sensory signals to infer modality-independent representations. To evaluate this hypothesis, we instantiate it in the form of a computational model that learns object shape representations from visual and/or haptic signals. The model uses a probabilistic grammar to characterize modality-independent representations of object shape, uses a computer graphics toolkit and a human hand simulator to map from object representations to visual and haptic features, respectively, and uses a Bayesian inference algorithm to infer modality-independent object representations from visual and/or haptic signals. Simulation results show that the model infers identical object representations when an object is viewed, grasped, or both. That is, the model’s percepts are modality invariant. We also report the results of an experiment in which different subjects rated the similarity of pairs of objects in different sensory conditions, and show that the model provides a very accurate account of subjects’ ratings. Conceptually, this research significantly contributes to our understanding of modality invariance, an important type of perceptual constancy, by demonstrating how modality-independent representations can be acquired and used. Methodologically, it provides an important contribution to cognitive modeling, particularly an emerging probabilistic language-of-thought approach, by showing how symbolic and statistical approaches can be combined in order to understand aspects of human perception.     

Spatiotemporal specificity of neuronal activity directs the modification of receptive fields in the developing retinotectal system

The precise temporal relation between pre- and postsynaptic activity modulates the strength of synaptic connections. Despite its extensive characterization in vivo and in vitro, the degree to which spike timing-dependent plasticity can shape receptive field properties is unclear. We use in vivo patch-clamp recordings of tectal neurons in developing Xenopus tadpoles to control the precise timing of action potentials with respect to the arrival of a subset of visual inputs evoked by local light stimulation on the retina. The pattern of visual inputs onto a tectal neuron was tracked over time by rapid reverse correlation mapping of receptive fields. Spike timing-dependent potentiation or depression of a subset of synapses reliably shifts the spatial receptive fields toward or away from the trained subregion of visual space, respectively. These results demonstrate that natural patterns of activity evoked by sensory stimuli play an instructive role in the developing nervous system.     

Information processing organs,mathematical mappings and self-organizing systems

A self-organizing system, which may be biological or man-made, adjusts itself in response to inputs from the surroundings. The input information is processed and transformed, so as to guide the system in accordance with a desired final state. The visual nervous system is considered, to illustrate some possible transformations or mappings, which may be employed by self-organizing systems. The mappings given as examples are linear, but there is evidence also for nonlinear mappings to explain the action of biological systems. The successive stages of adjustment in a self-organizing system can be treated as a feedback control process. Mathematically, feedback control of linear as well as nonlinear systems can be handled by using the principle of contraction mapping. The kind of control considered is flexible in the sense that a desired state of the system as a whole can be achieved through a variety of states of the individual parts. This leads to such questions as equivalence and classification which are also discussed in this paper.     

When Kinesthesia Becomes Visual: A Theoretical Justification for Executing Motor Tasks in Visual Space

Several experimental studies in the literature have shown that even when performing purely kinesthetic tasks, such as reaching for a kinesthetically felt target with a hidden hand, the brain reconstructs a visual representation of the movement. In our previous studies, however, we did not observe any role of a visual representation of the movement in a purely kinesthetic task. This apparent contradiction could be related to a fundamental difference between the studied tasks. In our study subjects used the same hand to both feel the target and to perform the movement, whereas in most other studies, pointing to a kinesthetic target consisted of pointing with one hand to the finger of the other, or to some other body part. We hypothesize, therefore, that it is the necessity of performing inter-limb transformations that induces a visual representation of purely kinesthetic tasks. To test this hypothesis we asked subjects to perform the same purely kinesthetic task in two conditions: INTRA and INTER. In the former they used the right hand to both perceive the target and to reproduce its orientation. In the latter, subjects perceived the target with the left hand and responded with the right. To quantify the use of a visual representation of the movement we measured deviations induced by an imperceptible conflict that was generated between visual and kinesthetic reference frames. Our hypothesis was confirmed by the observed deviations of responses due to the conflict in the INTER, but not in the INTRA, condition. To reconcile these observations with recent theories of sensori-motor integration based on maximum likelihood estimation, we propose here a new model formulation that explicitly considers the effects of covariance between sensory signals that are directly available and internal representations that are ‘reconstructed’ from those inputs through sensori-motor transformations.     

Otolith shape lends support to the sensory drive hypothesis in rockfishes

The sensory drive hypothesis proposes that environmental factors affect both signalling dynamics and the evolution of signals and receivers. Sound detection and equilibrium in marine fishes are senses dependent on the sagittae otoliths, whose morphological variability appears intrinsically linked to the environment. The aim of this study was to understand if and which environmental factors could be conditioning the evolution of this sensory structure, therefore lending support to the sensory drive hypothesis. Thus, we analysed the otolith shape of 42 rockfish species (Sebastes spp.) to test the potential associations with the phylogeny, biological (age), ecological (feeding habit and depth distribution) and biogeographical factors. The results showed strong differences in the otolith shapes of some species, noticeably influenced by ecological and biogeographical factors. Moreover, otolith shape was clearly conditioned by phylogeny, but with a strong environmental effect, cautioning about the use of this structure for the systematics of rockfishes or other marine fishes. However, our most relevant finding is that the data supported the sensory drive hypothesis as a force promoting the radiation of the genus Sebastes. This hypothesis holds that adaptive divergence in communication has significant influence relative to other life history traits. It has already been established in Sebastes for visual characters and organs; our results showed that it applies to otolith transformations as well (despite the clear influence of feeding and depth), expanding the scope of the hypothesis to other sensory structures.     

Effects of stimulus transformations on estimates of sensory neuron selectivity

Stimulus selectivity of sensory systems is often characterized by analyzing response-conditioned stimulus ensembles. However, in many cases these response-triggered stimulus sets have structure that is more complex than assumed. If not taken into account, when present it will bias the estimates of many simple statistics, and distort the estimated stimulus selectivity of a neural sensory system. We present an approach that mitigates these problems by modeling some of the response-conditioned stimulus structure as being generated by a set of transformations acting on a simple stimulus distribution. This approach corrects the estimates of key statistics and counters biases introduced by the transformations. In cases involving temporal spike jitter or spatial jitter of images, the main observed effects of transformations are blurring of the conditional mean and introduction of artefacts in the spectral decomposition of the conditional covariance matrix. We illustrate this approach by analyzing and correcting a set of model stimuli perturbed by temporal and spatial jitter. We apply the approach to neurophysiological data from the cricket cercal sensory system to correct the effects of temporal jitter. Action Editor: Matthew Wiener     

Some elementary considerations of neural models

The outputs of nervous systems (as expressed in motor activity) are viewed as mathematical transformations on the inputs which enter via the sensory nerves. Simple nerve-ganglion models are exhibited which theoretically account for the arithmetic computations necessary to expedite such transformations.