Eye-Head Coordination for Visual Cognitive Processing

We investigated coordinated movements between the eyes and head (“eye-head coordination”) in relation to vision for action. Several studies have measured eye and head movements during a single gaze shift, focusing on the mechanisms of motor control during eye-head coordination. However, in everyday life, gaze shifts occur sequentially and are accompanied by movements of the head and body. Under such conditions, visual cognitive processing influences eye movements and might also influence eye-head coordination because sequential gaze shifts include cycles of visual processing (fixation) and data acquisition (gaze shifts). In the present study, we examined how the eyes and head move in coordination during visual search in a large visual field. Subjects moved their eyes, head, and body without restriction inside a 360° visual display system. We found patterns of eye-head coordination that differed those observed in single gaze-shift studies. First, we frequently observed multiple saccades during one continuous head movement, and the contribution of head movement to gaze shifts increased as the number of saccades increased. This relationship between head movements and sequential gaze shifts suggests eye-head coordination over several saccade-fixation sequences; this could be related to cognitive processing because saccade-fixation cycles are the result of visual cognitive processing. Second, distribution bias of eye position during gaze fixation was highly correlated with head orientation. The distribution peak of eye position was biased in the same direction as head orientation. This influence of head orientation suggests that eye-head coordination is involved in gaze fixation, when the visual system processes retinal information. This further supports the role of eye-head coordination in visual cognitive processing.     

Ravens, Corvus corax, follow gaze direction of humans around obstacles

The ability to follow gaze (i.e. head and eye direction) has recently been shown for social mammals, particularly primates. In most studies, individuals could use gaze direction as a behavioural cue without understanding that the view of others may be different from their own. Here, we show that hand-raised ravens not only visually co-orient with the look-ups of a human experimenter but also reposition themselves to follow the experimenter's gaze around a visual barrier. Birds were capable of visual co-orientation already as fledglings but consistently tracked gaze direction behind obstacles not before six months of age. These results raise the possibility that sub-adult and adult ravens can project a line of sight for the other person into the distance. To what extent ravens may attribute mental significance to the visual behaviour of others is discussed.     

Evidence for on-line visual guidance during saccadic gaze shifts.

Rapid orientating movements of the eyes are believed to be controlled ballistically. The mechanism underlying this control is thought to involve a comparison between the desired displacement of the eye and an estimate of its actual position (obtained from the integration of the eye velocity signal). This study shows, however, that under certain circumstances fast gaze movements may be controlled quite differently and may involve mechanisms which use visual information to guide movements prospectively. Subjects were required to make large gaze shifts in yaw towards a target whose location and motion were unknown prior to movement onset. Six of those tested demonstrated remarkable accuracy when making gaze shifts towards a target that appeared during their ongoing movement. In fact their level of accuracy was not significantly different from that shown when they performed a 'remembered' gaze shift to a known stationary target (F3,15 = 0.15, p > 0.05). The lack of a stereotypical relationship between the skew of the gaze velocity profile and movement duration indicates that on-line modifications were being made. It is suggested that a fast route from the retina to the superior colliculus could account for this behaviour and that models of oculomotor control need to be updated.     

Joint Attention without Gaze Following: Human Infants and Their Parents Coordinate Visual Attention to Objects through Eye-Hand Coordination

The coordination of visual attention among social partners is central to many components of human behavior and human development. Previous research has focused on one pathway to the coordination of looking behavior by social partners, gaze following. The extant evidence shows that even very young infants follow the direction of another''s gaze but they do so only in highly constrained spatial contexts because gaze direction is not a spatially precise cue as to the visual target and not easily used in spatially complex social interactions. Our findings, derived from the moment-to-moment tracking of eye gaze of one-year-olds and their parents as they actively played with toys, provide evidence for an alternative pathway, through the coordination of hands and eyes in goal-directed action. In goal-directed actions, the hands and eyes of the actor are tightly coordinated both temporally and spatially, and thus, in contexts including manual engagement with objects, hand movements and eye movements provide redundant information about where the eyes are looking. Our findings show that one-year-olds rarely look to the parent''s face and eyes in these contexts but rather infants and parents coordinate looking behavior without gaze following by attending to objects held by the self or the social partner. This pathway, through eye-hand coupling, leads to coordinated joint switches in visual attention and to an overall high rate of looking at the same object at the same time, and may be the dominant pathway through which physically active toddlers align their looking behavior with a social partner.     

What Do Eye Gaze Metrics Tell Us about Motor Imagery?

Many of the brain structures involved in performing real movements also have increased activity during imagined movements or during motor observation, and this could be the neural substrate underlying the effects of motor imagery in motor learning or motor rehabilitation. In the absence of any objective physiological method of measurement, it is currently impossible to be sure that the patient is indeed performing the task as instructed. Eye gaze recording during a motor imagery task could be a possible way to “spy” on the activity an individual is really engaged in. The aim of the present study was to compare the pattern of eye movement metrics during motor observation, visual and kinesthetic motor imagery (VI, KI), target fixation, and mental calculation. Twenty-two healthy subjects (16 females and 6 males), were required to perform tests in five conditions using imagery in the Box and Block Test tasks following the procedure described by Liepert et al. Eye movements were analysed by a non-invasive oculometric measure (SMI RED250 system). Two parameters describing gaze pattern were calculated: the index of ocular mobility (saccade duration over saccade + fixation duration) and the number of midline crossings (i.e. the number of times the subjects gaze crossed the midline of the screen when performing the different tasks). Both parameters were significantly different between visual imagery and kinesthesic imagery, visual imagery and mental calculation, and visual imagery and target fixation. For the first time we were able to show that eye movement patterns are different during VI and KI tasks. Our results suggest gaze metric parameters could be used as an objective unobtrusive approach to assess engagement in a motor imagery task. Further studies should define how oculomotor parameters could be used as an indicator of the rehabilitation task a patient is engaged in.     

Eye movements: viewing the window of opportunity

When searching with our eyes, parallel programming of successive eye movements ensures that visual information arriving too late to alter the direction of one eye movement can still influence the direction of the next. Paradoxically, we can use random noise to probe the time period over which visual information influences where next to direct our gaze.     

Figure-ground activity in V1 and guidance of saccadic eye movements.

Every day we shift our gaze about 150.000 times mostly without noticing it. The direction of these gaze shifts are not random but directed by sensory information and internal factors. After each movement the eyes hold still for a brief moment so that visual information at the center of our gaze can be processed in detail. This means that visual information at the saccade target location is sufficient to accurately guide the gaze shift but yet is not sufficiently processed to be fully perceived. In this paper I will discuss the possible role of activity in the primary visual cortex (V1), in particular figure-ground activity, in oculo-motor behavior. Figure-ground activity occurs during the late response period of V1 neurons and correlates with perception. The strength of figure-ground responses predicts the direction and moment of saccadic eye movements. The superior colliculus, a gaze control center that integrates visual and motor signals, receives direct anatomical connections from V1. These projections may convey the perceptual information that is required for appropriate gaze shifts. In conclusion, figure-ground activity in V1 may act as an intermediate component linking visual and motor signals.     

Reading from a Head-Fixed Display during Walking: Adverse Effects of Gaze Stabilization Mechanisms

Reading performance during standing and walking was assessed for information presented on earth-fixed and head-fixed displays by determining the minimal duration during which a numerical time stimulus needed to be presented for 50% correct naming answers. Reading from the earth-fixed display was comparable during standing and walking, with optimal performance being attained for visual character sizes in the range of 0.2° to 1°. Reading from the head-fixed display was impaired for small (0.2-0.3°) and large (5°) visual character sizes, especially during walking. Analysis of head and eye movements demonstrated that retinal slip was larger during walking than during standing, but remained within the functional acuity range when reading from the earth-fixed display. The detrimental effects on performance of reading from the head-fixed display during walking could be attributed to loss of acuity resulting from large retinal slip. Because walking activated the angular vestibulo-ocular reflex, the resulting compensatory eye movements acted to stabilize gaze on the information presented on the earth-fixed display but destabilized gaze from the information presented on the head-fixed display. We conclude that the gaze stabilization mechanisms that normally allow visual performance to be maintained during physical activity adversely affect reading performance when the information is presented on a display attached to the head.     

Different programming modes of human saccadic eye movements as a function of stimulus eccentricity: Indications of a functional subdivision of the visual field

1. Voluntary saccadic eye movements were made toward flashes of light on the horizontal meridian, whose duration and distance from the point of fixation were varied; eye movements were measured using d.c.-electrooculography.—2. Targets within 10°–15° eccentricity are usually reached by one saccadic eye movement. When the eyes turn toward targets of more than 10°–15° eccentricity, the first saccadic eye movement falls short of the target by an angle usually not exceeding 10°. The presence of the image of the target off the fovea (visual error signal) subsequent to such an undershoot elicits, after a short interval, corrective saccades (usually one) which place the image of the target on the fovea. In the absence of a visual error signal, the probability of occurrence of corrective saccades is low, but it increases with greater target eccentricities. These observations suggest that there are different, eccentricity-dependent modes of programming saccadic eye movements.—3. Saccadic eye movements appear to be programmed in retinal coordinates. This conclusion is based on the observations that, irrespective of the initial position of the eyes in the orbit, a) there are different programming modes for eye movements to targets within and beyond 10°–15° from the fixation point, and b_ the maximum velocity of saccadic eye movements is always reached at 25° to 30° target eccentricity. —4. Distributions of latency and intersaccadic interval (ISI) are frequently multimodal, with a separation between modes of 30 to 40 msec. These observations suggest that saccadic eye movements are produced by mechanisms which, at a frequency of 30 Hz, process visual information. —5. Corrective saccades may occur after extremely short intervals (30 to 60 msec) regardless of whether or not a visual error signal is present; the eyes may not even come to a complete stop during these very short intersaccadic intervals. It is suggested that these corrective saccades are triggered by errors in the programming of the initial saccadic eye movements, and not by a visual error signal. —6. The exitence of different, eccentricity-dependent programming modes of saccadic eye movements, is further supported by anatomical, physiological, psychophysical, and neuropathological observations that suggest a dissociation of visual functions dependent on retinal eccentricity. Saccadic eye movements to targets more eccentric than 10°–15° appear to be executed by a mechanism involving the superior colliculus (perhaps independent of the visual cortex), whereas saccadic eye movements to less eccentric targets appear to depend on a mechanism involving the geniculo-cortical pathway (perhaps in collaboration with the superior colliculus).     

Coordination of eye and head movements in cats

Horizontal displacements of gaze in cats with unrestrained head were studied using the magnetic search coil method. Three types of eye-head coordination were found when cats oriented gaze towards visual targets. Maximal velocities of gaze, head and eye movements in orbits depend linearily on amplitudes of their displacements in the range of up to 20 degrees. Gaze velocity reached its top level in about 0.3 of complete time of movement execution. Data support the idea of saccadic-vestibular summation during coordinated eye-head movements in cats.     

Unique morphology of the human eye and its adaptive meaning: comparative studies on external morphology of the primate eye

In order to clarify the morphological uniqueness of the human eye and to obtain cues to understanding its adaptive significance, we compared the external morphology of the primate eye by measuring nearly half of all extant primate species. The results clearly showed exceptional features of the human eye: (1) the exposed white sclera is void of any pigmentation, (2) humans possess the largest ratio of exposed sclera in the eye outline, and (3) the eye outline is extraordinarily elongated in the horizontal direction. The close correlation of the parameters reflecting (2) and (3) with habitat type or body size of the species examined suggested that these two features are adaptations for extending the visual field by eyeball movement, especially in the horizontal direction. Comparison of eye coloration and facial coloration around the eye suggested that the dark coloration of exposed sclera of nonhuman primates is an adaptation to camouflage the gaze direction against other individuals and/or predators, and that the white sclera of the human eye is an adaptation to enhance the gaze signal. The uniqueness of human eye morphology among primates illustrates the remarkable difference between human and other primates in the ability to communicate using gaze signals.     

Predicting Aggressive Tendencies by Visual Attention Bias Associated with Hostile Emotions

The goal of the current study is to clarify the relationship between social information processing (e.g., visual attention to cues of hostility, hostility attribution bias, and facial expression emotion labeling) and aggressive tendencies. Thirty adults were recruited in the eye-tracking study that measured various components in social information processing. Baseline aggressive tendencies were measured using the Buss-Perry Aggression Questionnaire (AQ). Visual attention towards hostile objects was measured as the proportion of eye gaze fixation duration on cues of hostility. Hostility attribution bias was measured with the rating results for emotions of characters in the images. The results show that the eye gaze duration on hostile characters was significantly inversely correlated with the AQ score and less eye contact with an angry face. The eye gaze duration on hostile object was not significantly associated with hostility attribution bias, although hostility attribution bias was significantly positively associated with the AQ score. Our findings suggest that eye gaze fixation time towards non-hostile cues may predict aggressive tendencies.     

Iterative Fragmentation of Cognitive Maps in a Visual Imagery Task

It remains unclear whether spontaneous eye movements during visual imagery reflect the mental generation of a visual image (i.e. the arrangement of the component parts of a mental representation). To address this specificity, we recorded eye movements in an imagery task and in a phonological fluency (non-imagery) task, both consisting in naming French towns from long-term memory. Only in the condition of visual imagery the spontaneous eye positions reflected the geographic position of the towns evoked by the subjects. This demonstrates that eye positions closely reflect the mapping of mental images. Advanced analysis of gaze positions using the bi-dimensional regression model confirmed the spatial correlation of gaze and towns’ locations in every single individual in the visual imagery task and in none of the individuals when no imagery accompanied memory retrieval. In addition, the evolution of the bi-dimensional regression’s coefficient of determination revealed, in each individual, a process of generating several iterative series of a limited number of towns mapped with the same spatial distortion, despite different individual order of towns’ evocation and different individual mappings. Such consistency across subjects revealed by gaze (the mind’s eye) gives empirical support to theories postulating that visual imagery, like visual sampling, is an iterative fragmented processing.     

Kinematics of Visually-Guided Eye Movements

One of the hallmarks of an eye movement that follows Listing’s law is the half-angle rule that says that the angular velocity of the eye tilts by half the angle of eccentricity of the line of sight relative to primary eye position. Since all visually-guided eye movements in the regime of far viewing follow Listing’s law (with the head still and upright), the question about its origin is of considerable importance. Here, we provide theoretical and experimental evidence that Listing’s law results from a unique motor strategy that allows minimizing ocular torsion while smoothly tracking objects of interest along any path in visual space. The strategy consists in compounding conventional ocular rotations in meridian planes, that is in horizontal, vertical and oblique directions (which are all torsion-free) with small linear displacements of the eye in the frontal plane. Such compound rotation-displacements of the eye can explain the kinematic paradox that the fixation point may rotate in one plane while the eye rotates in other planes. Its unique signature is the half-angle law in the position domain, which means that the rotation plane of the eye tilts by half-the angle of gaze eccentricity. We show that this law does not readily generalize to the velocity domain of visually-guided eye movements because the angular eye velocity is the sum of two terms, one associated with rotations in meridian planes and one associated with displacements of the eye in the frontal plane. While the first term does not depend on eye position the second term does depend on eye position. We show that compounded rotation - displacements perfectly predict the average smooth kinematics of the eye during steady- state pursuit in both the position and velocity domain.     

Changes in vestibular postural response determined by information content of visual feedback

Electrical unipolar monoaural stimulation of the labyrinth led to body sway mainly on a frontal plane in normal human subjects in a standing position. Early and late stages of response with latencies of 120–200 and 200–500 msec respectively changing in size in accordance with conditions of visual control were distinguished in the stabilographic response. Maximum response was recorded when the eyes were closed. Response declined upon opening the eyes, fixing the gaze on a static target, and with visual feedback according to stabilograms. The early and late components declined by 10–20 and 50–70% respectively in all cases. Fixing the gaze, in darkness, on an illuminated light spot stationary in relation to the head had no effect on level of response. Once the expected direction of body sway had been imparted, a significant and almost identical decrease of 70–80% in both components took place with the gaze fixed, however. Early and late components of vestibulomotor response are thought to be mediated by regulatory mechasisms with differing time courses and functional connections.Institute of Research into Information Transmission, Academy of Sciences of the USSR, Moscow. Translated from Neirofiziologiya, Vo. 22, No. 1, pp. 80–87, January–February, 1990.     

Active vision and visual activation in area V4

During normal vision, the focus of gaze continually jumps from one important image feature to the next. In this issue of Neuron, Mazer and Gallant analyze neural activity in higher-level visual cortex during this kind of active visual exploration, and they demonstrate a localized enhancement of visual responses that predicts the target of the upcoming eye movement.     

Northern bald ibises follow others’ gaze into distant space but not behind barriers

Gaze following is the ability to use the visual orientation of others as a trigger to look in the same direction. Thereby, animals may either align their head and eye orientation with others (gaze following into distant space) or may even reposition themselves to look behind barriers impairing their perception (geometrical gaze following). It has been proposed that these two different modes are functionally and cognitively distinct, but experimental evidence for this claim is lacking. We here, to our knowledge, demonstrate for the first time, that adult animals may be capable of following gaze into distant space, but not geometrically around barriers. We tested Northern bald ibises (Geronticus eremita) for their ability to follow a conspecific''s gaze in two standard tasks. The birds readily looked up after seeing a model bird looking up; however, when seeing a model looking behind a barrier, they responded by looking at the barrier instead of walking around. These findings are in stark contrast to results obtained with great apes and corvids and provide the first experimental evidence, to our knowledge, for cognitive differences in gaze following tasks.     

A Method to Quantify Visual Information Processing in Children Using Eye Tracking

Visual problems that occur early in life can have major impact on a child''s development. Without verbal communication and only based on observational methods, it is difficult to make a quantitative assessment of a child''s visual problems. This limits accurate diagnostics in children under the age of 4 years and in children with intellectual disabilities. Here we describe a quantitative method that overcomes these problems. The method uses a remote eye tracker and a four choice preferential looking paradigm to measure eye movement responses to different visual stimuli. The child sits without head support in front of a monitor with integrated infrared cameras. In one of four monitor quadrants a visual stimulus is presented. Each stimulus has a specific visual modality with respect to the background, e.g., form, motion, contrast or color. From the reflexive eye movement responses to these specific visual modalities, output parameters such as reaction times, fixation accuracy and fixation duration are calculated to quantify a child''s viewing behavior. With this approach, the quality of visual information processing can be assessed without the use of communication. By comparing results with reference values obtained in typically developing children from 0-12 years, the method provides a characterization of visual information processing in visually impaired children. The quantitative information provided by this method can be advantageous for the field of clinical visual assessment and rehabilitation in multiple ways. The parameter values provide a good basis to: (i) characterize early visual capacities and consequently to enable early interventions; (ii) compare risk groups and follow visual development over time; and (iii), construct an individual visual profile for each child.     

Cell-type-specific synchronization of neural activity in FEF with V4 during attention

Shifts of gaze and shifts of attention are closely linked and it is debated whether they result from the same neural mechanisms. Both processes involve the frontal eye fields (FEF), an area which is also a source of top-down feedback to area V4 during covert attention. To test the relative contributions of oculomotor and attention-related FEF signals to such feedback, we recorded simultaneously from both areas in a covert attention task and in a saccade task. In the attention task, only visual and visuomovement FEF neurons showed enhanced responses, whereas movement cells were unchanged. Importantly, visual, but not movement or visuomovement cells, showed enhanced gamma frequency synchronization with activity in V4 during attention. Within FEF, beta synchronization was increased for movement cells during attention but was suppressed in the saccade task. These findings support the idea that the attentional modulation of visual processing is not mediated by movement neurons.     

Driving as night falls: the contribution of retinal flow and visual direction to the control of steering

We have the ability to locomote at high speeds, and we usually negotiate bends safely, even when visual information is degraded, for example, when driving at night. There are three sources of visual information that could support successful steering. An observer fixating a steering target that is eccentric to the current heading must rotate their gaze. The gaze rotation may be detected by using head and eye movement signals (extra-retinal direction: ERD) or their retinal counterpart, visual direction (VD). The gaze rotation also transforms the global retinal flow (RF) field, which may enable direct steering judgments. In this study, we manipulate VD and RF to determine their contribution toward steering a curved path in the presence of ERD. The results suggest a model that uses a weighted combination of all three information sources, but results also suggest that this weighting may change in reduced visibility, such as in low-light conditions.