Vision as oculomotor reward: cognitive contributions to the dynamic control of saccadic eye movements

Humans and other primates are equipped with a foveated visual system. As a consequence, we reorient our fovea to objects and targets in the visual field that are conspicuous or that we consider relevant or worth looking at. These reorientations are achieved by means of saccadic eye movements. Where we saccade to depends on various low-level factors such as a targets’ luminance but also crucially on high-level factors like the expected reward or a targets’ relevance for perception and subsequent behavior. Here, we review recent findings how the control of saccadic eye movements is influenced by higher-level cognitive processes. We first describe the pathways by which cognitive contributions can influence the neural oculomotor circuit. Second, we summarize what saccade parameters reveal about cognitive mechanisms, particularly saccade latencies, saccade kinematics and changes in saccade gain. Finally, we review findings on what renders a saccade target valuable, as reflected in oculomotor behavior. We emphasize that foveal vision of the target after the saccade can constitute an internal reward for the visual system and that this is reflected in oculomotor dynamics that serve to quickly and accurately provide detailed foveal vision of relevant targets in the visual field.     

Covert tracking: a combined ERP and fixational eye movement study

Attention can be directed to particular spatial locations, or to objects that appear at anticipated points in time. While most work has focused on spatial or temporal attention in isolation, we investigated covert tracking of smoothly moving objects, which requires continuous coordination of both. We tested two propositions about the neural and cognitive basis of this operation: first that covert tracking is a right hemisphere function, and second that pre-motor components of the oculomotor system are responsible for driving covert spatial attention during tracking. We simultaneously recorded event related potentials (ERPs) and eye position while participants covertly tracked dots that moved leftward or rightward at 12 or 20°/s. ERPs were sensitive to the direction of target motion. Topographic development in the leftward motion was a mirror image of the rightward motion, suggesting that both hemispheres contribute equally to covert tracking. Small shifts in eye position were also lateralized according to the direction of target motion, implying covert activation of the oculomotor system. The data addresses two outstanding questions about the nature of visuospatial tracking. First, covert tracking is reliant upon a symmetrical frontoparietal attentional system, rather than being right lateralized. Second, this same system controls both pursuit eye movements and covert tracking.     

Smooth Pursuit and Visual Occlusion: Active Inference and Oculomotor Control in Schizophrenia

This paper introduces a model of oculomotor control during the smooth pursuit of occluded visual targets. This model is based upon active inference, in which subjects try to minimise their (proprioceptive) prediction error based upon posterior beliefs about the hidden causes of their (exteroceptive) sensory input. Our model appeals to a single principle – the minimisation of variational free energy – to provide Bayes optimal solutions to the smooth pursuit problem. However, it tries to accommodate the cardinal features of smooth pursuit of partially occluded targets that have been observed empirically in normal subjects and schizophrenia. Specifically, we account for the ability of normal subjects to anticipate periodic target trajectories and emit pre-emptive smooth pursuit eye movements – prior to the emergence of a target from behind an occluder. Furthermore, we show that a single deficit in the postsynaptic gain of prediction error units (encoding the precision of posterior beliefs) can account for several features of smooth pursuit in schizophrenia: namely, a reduction in motor gain and anticipatory eye movements during visual occlusion, a paradoxical improvement in tracking unpredicted deviations from target trajectories and a failure to recognise and exploit regularities in the periodic motion of visual targets. This model will form the basis of subsequent (dynamic causal) models of empirical eye tracking measurements, which we hope to validate, using psychopharmacology and studies of schizophrenia.     

Cursive writing with smooth pursuit eye movements

The eyes never cease to move: ballistic saccades quickly turn the gaze toward peripheral targets, whereas smooth pursuit maintains moving targets on the fovea where visual acuity is best. Despite the oculomotor system being endowed with exquisite motor abilities, any attempt to generate smooth eye movements against a static background results in saccadic eye movements [1, 2]. Although exceptions to this rule have been reported [3-5], volitional control over smooth eye movements is at best rudimentary. Here, I introduce a novel, temporally modulated visual display, which, although static, sustains smooth eye movements in arbitrary directions. After brief training, participants gain volitional control over smooth pursuit eye movements and can generate digits, letters, words, or drawings at will. For persons deprived of limb movement, this offers a fast, creative, and personal means of linguistic and emotional expression.     

Which retinal and extra-retinal information is crucial for circular vection?

In contradistinction to conventional wisdom, we propose that retinal image slip of a visual scene (optokinetic pattern, OP) does not constitute the only crucial input for visually induced percepts of self-motion (vection). Instead, the hypothesis is investigated that there are three input factors: 1) OP retinal image slip, 2) motion of the ocular orbital shadows across the retinae, and 3) smooth pursuit eye movements (efference copy). To test this hypothesis, we visually induced percepts of sinusoidal rotatory self-motion (circular vection, CV) in the absence of vestibular stimulation. Subjects were presented with three concurrent stimuli: a large visual OP, a fixation point to be pursued with the eyes (both projected in superposition on a semi-circular screen), and a dark window frame placed close to the eyes to create artificial visual field boundaries that simulate ocular orbital rim boundary shadows, but which could be moved across the retinae independent from eye movements. In different combinations these stimuli were independently moved or kept stationary. When moved together (horizontally and sinusoidally around the subject's head), they did so in precise temporal synchrony at 0.05 Hz. The results show that the occurrence of CV requires retinal slip of the OP and/or relative motion between the orbital boundary shadows and the OP. On the other hand, CV does not develop when the two retinal slip signals equal each other (no relative motion) and concur with pursuit eye movements (as it is the case, e.g., when we follow with the eyes the motion of a target on a stationary visual scene). The findings were formalized in terms of a simulation model. In the model two signals coding relative motion between OP and head are fused and fed into the mechanism for CV, a visuo-oculomotor one, derived from OP retinal slip and eye movement efference copy, and a purely visual signal of relative motion between the orbital rims (head) and the OP. The latter signal is also used, together with a version of the oculomotor efference copy, for a mechanism that suppresses CV at a later stage of processing in conditions in which the retinal slip signals are self-generated by smooth pursuit eye movements.     

Anticipatory Smooth Eye Movements in Autism Spectrum Disorder

Smooth pursuit eye movements are important for vision because they maintain the line of sight on targets that move smoothly within the visual field. Smooth pursuit is driven by neural representations of motion, including a surprisingly strong influence of high-level signals representing expected motion. We studied anticipatory smooth eye movements (defined as smooth eye movements in the direction of expected future motion) produced by salient visual cues in a group of high-functioning observers with Autism Spectrum Disorder (ASD), a condition that has been associated with difficulties in either generating predictions, or translating predictions into effective motor commands. Eye movements were recorded while participants pursued the motion of a disc that moved within an outline drawing of an inverted Y-shaped tube. The cue to the motion path was a visual barrier that blocked the untraveled branch (right or left) of the tube. ASD participants showed strong anticipatory smooth eye movements whose velocity was the same as that of a group of neurotypical participants. Anticipatory smooth eye movements appeared on the very first cued trial, indicating that trial-by-trial learning was not responsible for the responses. These results are significant because they show that anticipatory capacities are intact in high-functioning ASD in cases where the cue to the motion path is highly salient and unambiguous. Once the ability to generate anticipatory pursuit is demonstrated, the study of the anticipatory responses with a variety of types of cues provides a window into the perceptual or cognitive processes that underlie the interpretation of events in natural environments or social situations.     

Blink Perturbation Effects on Saccades Evoked by Microstimulation of the Superior Colliculus

Current knowledge of saccade-blink interactions suggests that blinks have paradoxical effects on saccade generation. Blinks suppress saccade generation by attenuating the oculomotor drive command in structures like the superior colliculus (SC), but they also disinhibit the saccadic system by removing the potent inhibition of pontine omnipause neurons (OPNs). To better characterize these effects, we evoked the trigeminal blink reflex by delivering an air puff to one eye as saccades were evoked by sub-optimal stimulation of the SC. For every stimulation site, the peak and average velocities of stimulation with blink movements (SwBMs) were lower than stimulation-only saccades (SoMs), supporting the notion that the oculomotor drive is weakened in the presence of a blink. In contrast, the duration of the SwBMs was longer, consistent with the hypothesis that the blink-induced inhibition of the OPNs could prolong the window of time available for oculomotor commands to drive an eye movement. The amplitude of the SwBM could also be larger than the SoM amplitude obtained from the same site, particularly for cases in which blink-associated eye movements exhibited the slowest kinematics. The results are interpreted in terms of neural signatures of saccade-blink interactions.     

Saccadic adaptation to moving targets

Saccades are so called ballistic movements which are executed without online visual feedback. After each saccade the saccadic motor plan is modified in response to post-saccadic feedback with the mechanism of saccadic adaptation. The post-saccadic feedback is provided by the retinal position of the target after the saccade. If the target moves after the saccade, gaze may follow the moving target. In that case, the eyes are controlled by the pursuit system, a system that controls smooth eye movements. Although these two systems have in the past been considered as mostly independent, recent lines of research point towards many interactions between them. We were interested in the question if saccade amplitude adaptation is induced when the target moves smoothly after the saccade. Prior studies of saccadic adaptation have considered intra-saccadic target steps as learning signals. In the present study, the intra-saccadic target step of the McLaughlin paradigm of saccadic adaptation was replaced by target movement, and a post-saccadic pursuit of the target. We found that saccadic adaptation occurred in this situation, a further indication of an interaction of the saccadic system and the pursuit system with the aim of optimized eye movements.     

Colliding saccades evoked by frontal eye field stimulation: artifact or evidence for an oculomotor compensatory mechanism underlying double-step saccades?

 What happens when the goal is changed before the movement is executed? Both the double-step and colliding saccade paradigms address this issue as they introduce a discrepancy between the retinal images of targets in space and the commands generated by the oculomotor system necessary to attain those targets. To maintain spatial accuracy under such conditions, transformations must update ‘retinal error’ as eye position changes, and must also accommodate neural transmission delays in the system so that retinal and eye position information are temporally aligned. Different hypotheses have been suggested to account for these phenomena, based on observations of dissociable cortical and subcortical compensatory mechanisms. We now demonstrate how a single compensatory mechanism can be invoked to explain both double-step and colliding saccade paradigm results, based on the use of a damped signal of change in position that is used in both cases to update retinal error and, thereby, account for intervening movements. We conclude that the collision effect is not an artifact, but instead reveals a compensatory mechanism for saccades whose targets appear near the onset of a preceding saccade. Received: 14 February 1996/Accepted in revised form: 17 September 1996     

Directional anisotropies reveal a functional segregation of visual motion processing for perception and action

Human exhibits an anisotropy in direction perception: discrimination is superior when motion is around horizontal or vertical rather than diagonal axes. In contrast to the consistent directional anisotropy in perception, we found only small idiosyncratic anisotropies in smooth pursuit eye movements, a motor action requiring accurate discrimination of visual motion direction. Both pursuit and perceptual direction discrimination rely on signals from the middle temporal visual area (MT), yet analysis of multiple measures of MT neuronal responses in the macaque failed to provide evidence of a directional anisotropy. We conclude that MT represents different motion directions uniformly, and subsequent processing creates a directional anisotropy in pathways unique to perception. Our data support the hypothesis that, at least for visual motion, perception and action are guided by inputs from separate sensory streams. The directional anisotropy of perception appears to originate after the two streams have segregated and downstream from area MT.     

The composition of central programs subserving horizontal eye movements in man

A hypothesis is presented which describes, in biomechanical terms, the central programs underlying horizontal eye movements in man. It is suggested that eye movements are produced by means of programmed shifts of the so-called invariant muscle characteristics (static force vs angle of gaze). These shifts lead to a change of the equilibrium point resulting from the interaction of agnnist and antagonist muscles and, as a consequence, to movement and the attainment of a new position of gaze. A reciprocal or a coactivation command to agonist and antagonist muscles occurs when their characteristics shift with respect to the coordinate in the same or opposite directions, respectively. It is proposed that during pursuit and saccadic eye movements a supperposition of the both central commands occurs. During a saccade, the reciprocal command develops evenly up to a certain level. The initial and final levels of the reciprocal command dictate the respective position of gaze and therefore the size of the saccade. The coactivation command develops to a maximum level and is slowly switched off when the new position of gaze has been achieved. The magnitude of the coactivation command seems to be not connected with an absolute position of gaze. It provides probably a stability of the movement and, in particular, prevents overshoot and oscillation during the saccade. The same timing of these commands occurs during pursuit movements, but the magnitude of the coactivation command and the rates of the development of the both commands are less in this case and correlate with the velocity of the movement. This hypothesis enables the tension changes in the muscle during saccadic and pursuit movements to be simulated in qualitative accordance with unique experimental data obtained by Collins et al. (1975). The functional significance of superposition of these motor commands and similarity in the efferent organization of eye and limb movements are discussed.     

Smooth Pursuit Eye Movements Improve Temporal Resolution for Color Perception

Human observers see a single mixed color (yellow) when different colors (red and green) rapidly alternate. Accumulating evidence suggests that the critical temporal frequency beyond which chromatic fusion occurs does not simply reflect the temporal limit of peripheral encoding. However, it remains poorly understood how the central processing controls the fusion frequency. Here we show that the fusion frequency can be elevated by extra-retinal signals during smooth pursuit. This eye movement can keep the image of a moving target in the fovea, but it also introduces a backward retinal sweep of the stationary background pattern. We found that the fusion frequency was higher when retinal color changes were generated by pursuit-induced background motions than when the same retinal color changes were generated by object motions during eye fixation. This temporal improvement cannot be ascribed to a general increase in contrast gain of specific neural mechanisms during pursuit, since the improvement was not observed with a pattern flickering without changing position on the retina or with a pattern moving in the direction opposite to the background motion during pursuit. Our findings indicate that chromatic fusion is controlled by a cortical mechanism that suppresses motion blur. A plausible mechanism is that eye-movement signals change spatiotemporal trajectories along which color signals are integrated so as to reduce chromatic integration at the same locations (i.e., along stationary trajectories) on the retina that normally causes retinal blur during fixation.     

Segregation of object and background motion in visual area MT: effects of microstimulation on eye movements

To track a moving object, its motion must first be distinguished from that of the background. The center-surround properties of neurons in the middle temporal visual area (MT) may be important for signaling the relative motion between object and background. To test this, we microstimulated within MT and measured the effects on monkeys' eye movements to moving targets. We found that stimulation at "local motion" sites, where receptive fields possessed antagonistic surrounds, shifted pursuit in the preferred direction of the neurons, whereas stimulation at "wide-field motion" sites shifted pursuit in the opposite, or null, direction. We propose that activating wide-field sites simulated background motion, thus inducing a target motion signal in the opposite direction. Our results support the hypothesis that neuronal center-surround mechanisms contribute to the behavioral segregation of objects from the background.     

The representation of time for motor learning

We have identified factors that control precise motor timing by studying learning in smooth pursuit eye movements. Monkeys tracked a target that moved horizontally for a fixed time interval before changing direction through the addition of a vertical component of motion. After repeated presentations of the same target trajectory, infrequent probe trials of purely horizontal target motion evoked a vertical eye movement around the time when the change in target direction would have occurred. The pursuit system timed the vertical eye movement by keeping track of the duration of horizontal target motion and by measuring the distance the target traveled before changing direction, but not by learning the position in space where the target changed direction. We conclude that high temporal precision in motor output relies on multiple signals whose contributions to timing vary according to task requirements.     

Tuning Properties of MT and MSTd and Divisive Interactions for Eye-Movement Compensation

The primate brain intelligently processes visual information from the world as the eyes move constantly. The brain must take into account visual motion induced by eye movements, so that visual information about the outside world can be recovered. Certain neurons in the dorsal part of monkey medial superior temporal area (MSTd) play an important role in integrating information about eye movements and visual motion. When a monkey tracks a moving target with its eyes, these neurons respond to visual motion as well as to smooth pursuit eye movements. Furthermore, the responses of some MSTd neurons to the motion of objects in the world are very similar during pursuit and during fixation, even though the visual information on the retina is altered by the pursuit eye movement. We call these neurons compensatory pursuit neurons. In this study we develop a computational model of MSTd compensatory pursuit neurons based on physiological data from single unit studies. Our model MSTd neurons can simulate the velocity tuning of monkey MSTd neurons. The model MSTd neurons also show the pursuit compensation property. We find that pursuit compensation can be achieved by divisive interaction between signals coding eye movements and signals coding visual motion. The model generates two implications that can be tested in future experiments: (1) compensatory pursuit neurons in MSTd should have the same direction preference for pursuit and retinal visual motion; (2) there should be non-compensatory pursuit neurons that show opposite preferred directions of pursuit and retinal visual motion.     

Probability of Seeing Increases Saccadic Readiness

Associating movement directions or endpoints with monetary rewards or costs influences movement parameters in humans, and associating movement directions or endpoints with food reward influences movement parameters in non-human primates. Rewarded movements are facilitated relative to non-rewarded movements. The present study examined to what extent successful foveation facilitated saccadic eye movement behavior, with the hypothesis that foveation may constitute an informational reward. Human adults performed saccades to peripheral targets that either remained visible after saccade completion or were extinguished, preventing visual feedback. Saccades to targets that were systematically extinguished were slower and easier to inhibit than saccades to targets that afforded successful foveation, and this effect was modulated by the probability of successful foveation. These results suggest that successful foveation facilitates behavior, and that obtaining the expected sensory consequences of a saccadic eye movement may serve as a reward for the oculomotor system.     

Eye movements modulate visual receptive fields of V4 neurons

The receptive field, defined as the spatiotemporal selectivity of neurons to sensory stimuli, is central to our understanding of the neuronal mechanisms of perception. However, despite the fact that eye movements are critical during normal vision, the influence of eye movements on the structure of receptive fields has never been characterized. Here, we map the receptive fields of macaque area V4 neurons during saccadic eye movements and find that receptive fields are remarkably dynamic. Specifically, before the initiation of a saccadic eye movement, receptive fields shrink and shift towards the saccade target. These spatiotemporal dynamics may enhance information processing of relevant stimuli during the scanning of a visual scene, thereby assisting the selection of saccade targets and accelerating the analysis of the visual scene during free viewing.     

A model that integrates eye velocity commands to keep track of smooth eye displacements

Past results have reported conflicting findings on the oculomotor system’s ability to keep track of smooth eye movements in darkness. Whereas some results indicate that saccades cannot compensate for smooth eye displacements, others report that memory-guided saccades during smooth pursuit are spatially correct. Recently, it was shown that the amount of time before the saccade made a difference: short-latency saccades were retinotopically coded, whereas long-latency saccades were spatially coded. Here, we propose a model of the saccadic system that can explain the available experimental data. The novel part of this model consists of a delayed integration of efferent smooth eye velocity commands. Two alternative physiologically realistic neural mechanisms for this integration stage are proposed. Model simulations accurately reproduced prior findings. Thus, this model reconciles the earlier contradictory reports from the literature about compensation for smooth eye movements before saccades because it involves a slow integration process. Action Editor: Jonathan D. Victor     

Control and modulation of canal driven vestibulo-ocular reflex.

It has been well known that the canal driven vestibulo-ocular reflex (VOR) is controlled and modulated through the central nervous system by external sensory information (e.g. visual, otolithic and somatosensory inputs) and by mental conditions. Because the origin of retinal image motion exists both in the subjects (eye, head and body motions) and in the external world (object motion), the head motion should be canceled and/or the object should be followed by smooth eye movements. Human has developed a lot of central nervous mechanisms for smooth eye movements (e.g. VOR, optokinetic reflex and smooth pursuit eye movements). These mechanisms are thought to work for the purpose of better seeing. Distinct mechanism will work in appropriate self motion and/or object motion. As the results, whole mechanisms are controlled in a purpose-directed manner. This can be achieved by a self-organizing holistic system. Holistic system is very useful for understanding human oculomotor behavior.     

Mechanisms of Elementary Eye Movements as a Tracking System

In the study of eye-movement mechanisms, the most prevalent hypotheses are those that explain these mechanisms as a programmed system. The assumption is that before the eye accomplishes any movement, a program must be constituted in a control center to define the characteristics of the movement. It is claimed that programming governs all aspects of the oculomotor system: those determining the path of the gaze, that is, the route followed in observing objects, and those controlling elementary movements. For example, it is assumed that before the eye makes a saccade from one fixation point to another, a program should already have been compiled defining the direction, amplitude, and speed of this signal. This program is formed in the latency period preceding the saccade.