A simple translation in cortical log-coordinates may account for the pattern of saccadic localization errors

During saccadic eye movements, the visual world shifts rapidly across the retina. Perceptual continuity is thought to be maintained by active neural mechanisms that compensate for this displacement, bringing the presaccadic scene into a postsaccadic reference frame. Because of this active mechanism, objects appearing briefly around the time of the saccade are perceived at erroneous locations, a phenomenon called perisaccadic mislocalization. The position and direction of localization errors can inform us about the different reference frames involved. It has been found, for example, that errors are not simply made in the direction of the saccade but directed toward the saccade target, indicating that the compensatory mechanism involves spatial compression rather than translation. A recent study confirmed that localization errors also occur in the direction orthogonal to saccade direction, but only for eccentricities far from the fovea, beyond the saccade target. This spatially specific pattern of distortion cannot be explained by a simple compression of space around the saccade target. Here I show that a change of reference frames (i.e., translation) in cortical (logarithmic) coordinates, taking into account the cortical magnification factor, can accurately predict these spatial patterns of mislocalization. The flashed object projects onto the cortex in presaccadic (fovea-centered) coordinates but is perceived in postsaccadic (target-centered) coordinates.     

Two distinct visual motion mechanisms for smooth pursuit: evidence from individual differences

Smooth-pursuit eye velocity to a moving target is more accurate after an initial catch-up saccade than before, an enhancement that is poorly understood. We present an individual-differences-based method for identifying mechanisms underlying a physiological response and use it to test whether visual motion signals driving pursuit differ pre- and postsaccade. Correlating moment-to-moment measurements of pursuit over time with two psychophysical measures of speed estimation during fixation, we find two independent associations across individuals. Presaccadic pursuit acceleration is predicted by the precision of low-level (motion-energy-based) speed estimation, and postsaccadic pursuit precision is predicted by the precision of high-level (position-tracking) speed estimation. These results provide evidence that a low-level motion signal influences presaccadic acceleration and an independent high-level motion signal influences postsaccadic precision, thus presenting a plausible mechanism for postsaccadic enhancement of pursuit.     

Perisaccadic mislocalization orthogonal to saccade direction

Saccadic eye movements transiently distort perceptual space. Visual objects flashed shortly before or during a saccade are mislocalized along the saccade direction, resembling a compression of space around the saccade target. These mislocalizations reflect transient errors of processes that construct spatial stability across eye movements. They may arise from errors of reference signals associated with saccade direction and amplitude or from visual or visuomotor remapping processes focused on the saccade target's position. The second case would predict apparent position shifts toward the target also in directions orthogonal to the saccade. We report that such orthogonal mislocalization indeed occurs. Surprisingly, however, the orthogonal mislocalization is restricted to only part of the visual field. This part comprises distant positions in saccade direction but does not depend on the target's position. Our findings can be explained by a combination of directional and positional reference signals that varies in time course across the visual field.     

Attention controls multisensory perception via two distinct mechanisms at different levels of the cortical hierarchy

To form a percept of the multisensory world, the brain needs to integrate signals from common sources weighted by their reliabilities and segregate those from independent sources. Previously, we have shown that anterior parietal cortices combine sensory signals into representations that take into account the signals’ causal structure (i.e., common versus independent sources) and their sensory reliabilities as predicted by Bayesian causal inference. The current study asks to what extent and how attentional mechanisms can actively control how sensory signals are combined for perceptual inference. In a pre- and postcueing paradigm, we presented observers with audiovisual signals at variable spatial disparities. Observers were precued to attend to auditory or visual modalities prior to stimulus presentation and postcued to report their perceived auditory or visual location. Combining psychophysics, functional magnetic resonance imaging (fMRI), and Bayesian modelling, we demonstrate that the brain moulds multisensory inference via two distinct mechanisms. Prestimulus attention to vision enhances the reliability and influence of visual inputs on spatial representations in visual and posterior parietal cortices. Poststimulus report determines how parietal cortices flexibly combine sensory estimates into spatial representations consistent with Bayesian causal inference. Our results show that distinct neural mechanisms control how signals are combined for perceptual inference at different levels of the cortical hierarchy.

A combination of psychophysics, computational modelling and fMRI reveals novel insights into how the brain controls the binding of information across the senses, such as the voice and lip movements of a speaker.     

Visual attention and stability

In the present review, we address the relationship between attention and visual stability. Even though with each eye, head and body movement the retinal image changes dramatically, we perceive the world as stable and are able to perform visually guided actions. However, visual stability is not as complete as introspection would lead us to believe. We attend to only a few items at a time and stability is maintained only for those items. There appear to be two distinct mechanisms underlying visual stability. The first is a passive mechanism: the visual system assumes the world to be stable, unless there is a clear discrepancy between the pre- and post-saccadic image of the region surrounding the saccade target. This is related to the pre-saccadic shift of attention, which allows for an accurate preview of the saccade target. The second is an active mechanism: information about attended objects is remapped within retinotopic maps to compensate for eye movements. The locus of attention itself, which is also characterized by localized retinotopic activity, is remapped as well. We conclude that visual attention is crucial in our perception of a stable world.     

Visual stability

Our vision remains stable even though the movements of our eyes, head and bodies create a motion pattern on the retina. One of the most important, yet basic, feats of the visual system is to correctly determine whether this retinal motion is owing to real movement in the world or rather our own self-movement. This problem has occupied many great thinkers, such as Descartes and Helmholtz, at least since the time of Alhazen. This theme issue brings together leading researchers from animal neurophysiology, clinical neurology, psychophysics and cognitive neuroscience to summarize the state of the art in the study of visual stability. Recently, there has been significant progress in understanding the limits of visual stability in humans and in identifying many of the brain circuits involved in maintaining a stable percept of the world. Clinical studies and new experimental methods, such as transcranial magnetic stimulation, now make it possible to test the causal role of different brain regions in creating visual stability and also allow us to measure the consequences when the mechanisms of visual stability break down.     

Experimental Test of Spatial Updating Models for Monkey Eye-Head Gaze Shifts

How the brain maintains an accurate and stable representation of visual target locations despite the occurrence of saccadic gaze shifts is a classical problem in oculomotor research. Here we test and dissociate the predictions of different conceptual models for head-unrestrained gaze-localization behavior of macaque monkeys. We adopted the double-step paradigm with rapid eye-head gaze shifts to measure localization accuracy in response to flashed visual stimuli in darkness. We presented the second target flash either before (static), or during (dynamic) the first gaze displacement. In the dynamic case the brief visual flash induced a small retinal streak of up to about 20 deg at an unpredictable moment and retinal location during the eye-head gaze shift, which provides serious challenges for the gaze-control system. However, for both stimulus conditions, monkeys localized the flashed targets with accurate gaze shifts, which rules out several models of visuomotor control. First, these findings exclude the possibility that gaze-shift programming relies on retinal inputs only. Instead, they support the notion that accurate eye-head motor feedback updates the gaze-saccade coordinates. Second, in dynamic trials the visuomotor system cannot rely on the coordinates of the planned first eye-head saccade either, which rules out remapping on the basis of a predictive corollary gaze-displacement signal. Finally, because gaze-related head movements were also goal-directed, requiring continuous access to eye-in-head position, we propose that our results best support a dynamic feedback scheme for spatial updating in which visuomotor control incorporates accurate signals about instantaneous eye- and head positions rather than relative eye- and head displacements.     

Spatiotopic visual maps revealed by saccadic adaptation in humans

Saccadic adaptation [1] is a powerful experimental paradigm to probe the mechanisms of eye movement control and spatial vision, in which saccadic amplitudes change in response to false visual feedback. The adaptation occurs primarily in the motor system [2, 3], but there is also evidence for visual adaptation, depending on the size and the permanence of the postsaccadic error [4-7]. Here we confirm that adaptation has a strong visual component and show that the visual component of the adaptation is spatially selective in external, not retinal coordinates. Subjects performed?a memory-guided, double-saccade, outward-adaptation task designed to maximize visual adaptation and to dissociate the visual and motor corrections. When the memorized saccadic target was in the same position (in external space) as that used in the adaptation training, saccade targeting was strongly influenced by adaptation (even if not matched in retinal or cranial position), but when in the same retinal or cranial but different external spatial position, targeting was unaffected by adaptation, demonstrating unequivocal spatiotopic selectivity. These results point to the existence of a spatiotopic neural representation for eye movement control that adapts in response to saccade error signals.     

Eye movements modulate visual receptive fields of V4 neurons

The receptive field, defined as the spatiotemporal selectivity of neurons to sensory stimuli, is central to our understanding of the neuronal mechanisms of perception. However, despite the fact that eye movements are critical during normal vision, the influence of eye movements on the structure of receptive fields has never been characterized. Here, we map the receptive fields of macaque area V4 neurons during saccadic eye movements and find that receptive fields are remarkably dynamic. Specifically, before the initiation of a saccadic eye movement, receptive fields shrink and shift towards the saccade target. These spatiotemporal dynamics may enhance information processing of relevant stimuli during the scanning of a visual scene, thereby assisting the selection of saccade targets and accelerating the analysis of the visual scene during free viewing.     

Perisaccadic Remapping and Rescaling of Visual Responses in Macaque Superior Colliculus

Visual neurons have spatial receptive fields that encode the positions of objects relative to the fovea. Because foveate animals execute frequent saccadic eye movements, this position information is constantly changing, even though the visual world is generally stationary. Interestingly, visual receptive fields in many brain regions have been found to exhibit changes in strength, size, or position around the time of each saccade, and these changes have often been suggested to be involved in the maintenance of perceptual stability. Crucial to the circuitry underlying perisaccadic changes in visual receptive fields is the superior colliculus (SC), a brainstem structure responsible for integrating visual and oculomotor signals. In this work we have studied the time-course of receptive field changes in the SC. We find that the distribution of the latencies of SC responses to stimuli placed outside the fixation receptive field is bimodal: The first mode is comprised of early responses that are temporally locked to the onset of the visual probe stimulus and stronger for probes placed closer to the classical receptive field. We suggest that such responses are therefore consistent with a perisaccadic rescaling, or enhancement, of weak visual responses within a fixed spatial receptive field. The second mode is more similar to the remapping that has been reported in the cortex, as responses are time-locked to saccade onset and stronger for stimuli placed in the postsaccadic receptive field location. We suggest that these two temporal phases of spatial updating may represent different sources of input to the SC.     

Saccadic adaptation to moving targets

Saccades are so called ballistic movements which are executed without online visual feedback. After each saccade the saccadic motor plan is modified in response to post-saccadic feedback with the mechanism of saccadic adaptation. The post-saccadic feedback is provided by the retinal position of the target after the saccade. If the target moves after the saccade, gaze may follow the moving target. In that case, the eyes are controlled by the pursuit system, a system that controls smooth eye movements. Although these two systems have in the past been considered as mostly independent, recent lines of research point towards many interactions between them. We were interested in the question if saccade amplitude adaptation is induced when the target moves smoothly after the saccade. Prior studies of saccadic adaptation have considered intra-saccadic target steps as learning signals. In the present study, the intra-saccadic target step of the McLaughlin paradigm of saccadic adaptation was replaced by target movement, and a post-saccadic pursuit of the target. We found that saccadic adaptation occurred in this situation, a further indication of an interaction of the saccadic system and the pursuit system with the aim of optimized eye movements.     

Visual stability and the motion aftereffect: a psychophysical study revealing spatial updating

Eye movements create an ever-changing image of the world on the retina. In particular, frequent saccades call for a compensatory mechanism to transform the changing visual information into a stable percept. To this end, the brain presumably uses internal copies of motor commands. Electrophysiological recordings of visual neurons in the primate lateral intraparietal cortex, the frontal eye fields, and the superior colliculus suggest that the receptive fields (RFs) of special neurons shift towards their post-saccadic positions before the onset of a saccade. However, the perceptual consequences of these shifts remain controversial. We wanted to test in humans whether a remapping of motion adaptation occurs in visual perception.The motion aftereffect (MAE) occurs after viewing of a moving stimulus as an apparent movement to the opposite direction. We designed a saccade paradigm suitable for revealing pre-saccadic remapping of the MAE. Indeed, a transfer of motion adaptation from pre-saccadic to post-saccadic position could be observed when subjects prepared saccades. In the remapping condition, the strength of the MAE was comparable to the effect measured in a control condition (33±7% vs. 27±4%). Contrary, after a saccade or without saccade planning, the MAE was weak or absent when adaptation and test stimulus were located at different retinal locations, i.e. the effect was clearly retinotopic. Regarding visual cognition, our study reveals for the first time predictive remapping of the MAE but no spatiotopic transfer across saccades. Since the cortical sites involved in motion adaptation in primates are most likely the primary visual cortex and the middle temporal area (MT/V5) corresponding to human MT, our results suggest that pre-saccadic remapping extends to these areas, which have been associated with strict retinotopy and therefore with classical RF organization. The pre-saccadic transfer of visual features demonstrated here may be a crucial determinant for a stable percept despite saccades.     

Apparent Motion from Outside the Visual Field,Retinotopic Cortices May Register Extra-Retinal Positions

Observers made a saccade between two fixation markers while a probe was flashed sequentially at two locations on a side screen. The first probe was presented in the far periphery just within the observer''s visual field. This target was extinguished and the observers made a large saccade away from the probe, which would have left it far outside the visual field if it had still been present. The second probe was then presented, displaced from the first in the same direction as the eye movement and by about the same distance as the saccade step. Because both eyes and probes shifted by similar amounts, there was little or no shift between the first and second probe positions on the retina. Nevertheless, subjects reported seeing motion corresponding to the spatial displacement not the retinal displacement. When the second probe was presented, the effective location of the first probe lay outside the visual field demonstrating that apparent motion can be seen from a location outside the visual field to a second location inside the visual field. Recent physiological results suggest that target locations are “remapped” on retinotopic representations to correct for the effects of eye movements. Our results suggest that the representations on which this remapping occurs include locations that fall beyond the limits of the retina.     

Cortical Hierarchies Perform Bayesian Causal Inference in Multisensory Perception

To form a veridical percept of the environment, the brain needs to integrate sensory signals from a common source but segregate those from independent sources. Thus, perception inherently relies on solving the “causal inference problem.” Behaviorally, humans solve this problem optimally as predicted by Bayesian Causal Inference; yet, the underlying neural mechanisms are unexplored. Combining psychophysics, Bayesian modeling, functional magnetic resonance imaging (fMRI), and multivariate decoding in an audiovisual spatial localization task, we demonstrate that Bayesian Causal Inference is performed by a hierarchy of multisensory processes in the human brain. At the bottom of the hierarchy, in auditory and visual areas, location is represented on the basis that the two signals are generated by independent sources (= segregation). At the next stage, in posterior intraparietal sulcus, location is estimated under the assumption that the two signals are from a common source (= forced fusion). Only at the top of the hierarchy, in anterior intraparietal sulcus, the uncertainty about the causal structure of the world is taken into account and sensory signals are combined as predicted by Bayesian Causal Inference. Characterizing the computational operations of signal interactions reveals the hierarchical nature of multisensory perception in human neocortex. It unravels how the brain accomplishes Bayesian Causal Inference, a statistical computation fundamental for perception and cognition. Our results demonstrate how the brain combines information in the face of uncertainty about the underlying causal structure of the world.     

Fixational Eye Movements in the Earliest Stage of Metazoan Evolution

All known photoreceptor cells adapt to constant light stimuli, fading the retinal image when exposed to an immobile visual scene. Counter strategies are therefore necessary to prevent blindness, and in mammals this is accomplished by fixational eye movements. Cubomedusae occupy a key position for understanding the evolution of complex visual systems and their eyes are assumedly subject to the same adaptive problems as the vertebrate eye, but lack motor control of their visual system. The morphology of the visual system of cubomedusae ensures a constant orientation of the eyes and a clear division of the visual field, but thereby also a constant retinal image when exposed to stationary visual scenes. Here we show that bell contractions used for swimming in the medusae refresh the retinal image in the upper lens eye of Tripedalia cystophora. This strongly suggests that strategies comparable to fixational eye movements have evolved at the earliest metazoan stage to compensate for the intrinsic property of the photoreceptors. Since the timing and amplitude of the rhopalial movements concur with the spatial and temporal resolution of the eye it circumvents the need for post processing in the central nervous system to remove image blur.     

Sensory processing of motor inaccuracy depends on previously performed movement and on subsequent motor corrections: a study of the saccadic system

When goal-directed movements are inaccurate, two responses are generated by the brain: a fast motor correction toward the target and an adaptive motor recalibration developing progressively across subsequent trials. For the saccadic system, there is a clear dissociation between the fast motor correction (corrective saccade production) and the adaptive motor recalibration (primary saccade modification). Error signals used to trigger corrective saccades and to induce adaptation are based on post-saccadic visual feedback. The goal of this study was to determine if similar or different error signals are involved in saccadic adaptation and in corrective saccade generation. Saccadic accuracy was experimentally altered by systematically displacing the visual target during motor execution. Post-saccadic error signals were studied by manipulating visual information in two ways. First, the duration of the displaced target after primary saccade termination was set at 15, 50, 100 or 800 ms in different adaptation sessions. Second, in some sessions, the displaced target was followed by a visual mask that interfered with visual processing. Because they rely on different mechanisms, the adaptation of reactive saccades and the adaptation of voluntary saccades were both evaluated. We found that saccadic adaptation and corrective saccade production were both affected by the manipulations of post-saccadic visual information, but in different ways. This first finding suggests that different types of error signal processing are involved in the induction of these two motor corrections. Interestingly, voluntary saccades required a longer duration of post-saccadic target presentation to reach the same amount of adaptation as reactive saccades. Finally, the visual mask interfered with the production of corrective saccades only during the voluntary saccades adaptation task. These last observations suggest that post-saccadic perception depends on the previously performed action and that the differences between saccade categories of motor correction and adaptation occur at an early level of visual processing.     

The influence of eye dominance on saccade characteristics and slow presaccadic potentials

Characteristics of saccades and presaccadic slow potentials were studied in 36 right-handed men with right (the RE group) and left (the LE group) eye dominance. Three light-emitting diodes located in the center of the visual field (the central fixation stimulus, CFS) and 10 deg to the left and to the right of the center (peripheral stimuli, PSs) were used for stimulation. The subjects performed a task with simple saccades to a PS and a task with antisaccades to the horizontal mirror position of the PS. Monopolar EEGs at 19 derivations and electrooculograms (EOGs) were recorded. Back averaging of the EEG time-locked to the PS onset or the saccade onset was used to obtain slow presaccadic potentials. The saccade characteristics in the RE and LE groups were similar. Differences between them were found only in the antisaccade task. The amplitude of negative presaccadic potentials (NPPs) time-locked to the PS in the frontal cortex was lower in the LE group compared to the RE group. Analysis of potentials time-locked to the saccade onset showed that changes in the slow potentials during the last 50 s before the saccade depended on the saccade direction and reflected the activation of the hemisphere opposite to the saccade direction. The activation of the right hemisphere before left-side saccades was higher in the LE than the RE group. In addition, the amplitude of NPPs was decreased in the frontal area and increased in the left posterior temporal area in the LE group compared to the RE group. The obtained results indicate that the involvement of the frontal cortex in cognitive and motor processes is decreased in subjects with the left eye dominance.     

Vision as oculomotor reward: cognitive contributions to the dynamic control of saccadic eye movements

Humans and other primates are equipped with a foveated visual system. As a consequence, we reorient our fovea to objects and targets in the visual field that are conspicuous or that we consider relevant or worth looking at. These reorientations are achieved by means of saccadic eye movements. Where we saccade to depends on various low-level factors such as a targets’ luminance but also crucially on high-level factors like the expected reward or a targets’ relevance for perception and subsequent behavior. Here, we review recent findings how the control of saccadic eye movements is influenced by higher-level cognitive processes. We first describe the pathways by which cognitive contributions can influence the neural oculomotor circuit. Second, we summarize what saccade parameters reveal about cognitive mechanisms, particularly saccade latencies, saccade kinematics and changes in saccade gain. Finally, we review findings on what renders a saccade target valuable, as reflected in oculomotor behavior. We emphasize that foveal vision of the target after the saccade can constitute an internal reward for the visual system and that this is reflected in oculomotor dynamics that serve to quickly and accurately provide detailed foveal vision of relevant targets in the visual field.     

Looking for Discriminating Is Different from Looking for Looking’s Sake

Recent studies provide evidence for task-specific influences on saccadic eye movements. For instance, saccades exhibit higher peak velocity when the task requires coordinating eye and hand movements. The current study shows that the need to process task-relevant visual information at the saccade endpoint can be, in itself, sufficient to cause such effects. In this study, participants performed a visual discrimination task which required a saccade for successful completion. We compared the characteristics of these task-related saccades to those of classical target-elicited saccades, which required participants to fixate a visual target without performing a discrimination task. The results show that task-related saccades are faster and initiated earlier than target-elicited saccades. Differences between both saccade types are also noted in their saccade reaction time distributions and their main sequences, i.e., the relationship between saccade velocity, duration, and amplitude.     

Bayesian processing of vestibular information

Complex self-motion stimulations in the dark can be powerfully disorienting and can create illusory motion percepts. In the absence of visual cues, the brain has to use angular and linear acceleration information provided by the vestibular canals and the otoliths, respectively. However, these sensors are inaccurate and ambiguous. We propose that the brain processes these signals in a statistically optimal fashion, reproducing the rules of Bayesian inference. We also suggest that this processing is related to the statistics of natural head movements. This would create a perceptual bias in favour of low velocity and acceleration. We have constructed a Bayesian model of self-motion perception based on these assumptions. Using this model, we have simulated perceptual responses to centrifugation and off-vertical axis rotation and obtained close agreement with experimental findings. This demonstrates how Bayesian inference allows to make a quantitative link between sensor noise and ambiguities, statistics of head movement, and the perception of self-motion.