Iterative Fragmentation of Cognitive Maps in a Visual Imagery Task

It remains unclear whether spontaneous eye movements during visual imagery reflect the mental generation of a visual image (i.e. the arrangement of the component parts of a mental representation). To address this specificity, we recorded eye movements in an imagery task and in a phonological fluency (non-imagery) task, both consisting in naming French towns from long-term memory. Only in the condition of visual imagery the spontaneous eye positions reflected the geographic position of the towns evoked by the subjects. This demonstrates that eye positions closely reflect the mapping of mental images. Advanced analysis of gaze positions using the bi-dimensional regression model confirmed the spatial correlation of gaze and towns’ locations in every single individual in the visual imagery task and in none of the individuals when no imagery accompanied memory retrieval. In addition, the evolution of the bi-dimensional regression’s coefficient of determination revealed, in each individual, a process of generating several iterative series of a limited number of towns mapped with the same spatial distortion, despite different individual order of towns’ evocation and different individual mappings. Such consistency across subjects revealed by gaze (the mind’s eye) gives empirical support to theories postulating that visual imagery, like visual sampling, is an iterative fragmented processing.     

What Happens during the Stimulus Onset Asynchrony in the Dot-Probe Task? Exploring the Role of Eye Movements in the Assessment of Attentional Biases

The dot-probe paradigm is one of the most often used paradigms to investigate attentional biases towards emotional information. However, a large number of the dot-probe studies so far used a long stimulus onset asynchrony allowing for eye movements to occur, which might increase the error variance. This study aimed at addressing this methodological issue by varying the instructions with regard to the gaze behavior and calculating the reaction time (RT) bias score (i.e., RTs for targets presented at the location of the emotional compared to the neutral stimulus) separately for trials with eye movements and trials without eye movements. Results of Experiment 1 (using typical instructions, i.e., instructions that are lenient with regard to eye movements) showed an RT bias, but only in the trials without eye movements The overall RT bias (calculated “blind” for eye movements) was non-significant. In Experiment 2, stricter instructions and small changes in the procedure led to a sharp decrease in the number of eye movements, such that both the RT bias in the trials without eye movements as well as the RT bias across all trials was significant.     

A note on visual integrative mechanisms

In a system as complex and as effective as the eye, the cooperative interaction of different mechanisms may be taken as axiomatic. With this as a starting point, various visual phenomena are considered, such as short term memory, eye movements, and flicker fusion. Simple data on mean values lend support to the proposition that the spatial and temporal characteristics of these phenomena are matched with one another. The significance of this for a mechanism of vision is discussed.     

Frame of reference transformations in motion perception during smooth pursuit eye movements

Smooth pursuit eye movements change the retinal image velocity of objects in the visual field. In order to change from a retinocentric frame of reference into a head-centric one, the visual system has to take the eye movements into account. Studies on motion perception during smooth pursuit eye movements have measured either perceived speed or perceived direction during smooth pursuit to investigate this frame of reference transformation, but never both at the same time. We devised a new velocity matching task, in which participants matched both perceived speed and direction during fixation to that during pursuit. In Experiment 1, the velocity matches were determined for a range of stimulus directions, with the head-centric stimulus speed kept constant. In Experiment 2, the retinal stimulus speed was kept approximately constant, with the same range of stimulus directions. In both experiments, the velocity matches for all directions were shifted against the pursuit direction, suggesting an incomplete transformation of the frame of reference. The degree of compensation was approximately constant across stimulus direction. We fitted the classical linear model, the model of Turano and Massof (2001) and that of Freeman (2001) to the velocity matches. The model of Turano and Massof fitted the velocity matches best, but the differences between de model fits were quite small. Evaluation of the models and comparison to a few alternatives suggests that further specification of the potential effect of retinal image characteristics on the eye movement signal is needed.     

Memory and prediction in natural gaze control

In addition to stimulus properties and task factors, memory is an important determinant of the allocation of attention and gaze in the natural world. One way that the role of memory is revealed is by predictive eye movements. Both smooth pursuit and saccadic eye movements demonstrate predictive effects based on previous experience. We have previously shown that unskilled subjects make highly accurate predictive saccades to the anticipated location of a ball prior to a bounce in a virtual racquetball setting. In this experiment, we examined this predictive behaviour. We asked whether the period after the bounce provides subjects with visual information about the ball trajectory that is used to programme the pursuit movement initiated when the ball passes through the fixation point. We occluded a 100 ms period of the ball''s trajectory immediately after the bounce, and found very little effect on the subsequent pursuit movement. Subjects did not appear to modify their strategy to prolong the fixation. Neither were we able to find an effect on interception performance. Thus, it is possible that the occluded trajectory information is not critical for subsequent pursuit, and subjects may use an estimate of the ball''s trajectory to programme pursuit. These results provide further support for the role of memory in eye movements.     

The Relation between Geometry and Time in Mental Actions

Mental imagery is a cognitive tool that helps humans take decisions by simulating past and future events. The hypothesis has been advanced that there is a functional equivalence between actual and mental movements. Yet, we do not know whether there are any limitations to its validity even in terms of some fundamental features of actual movements, such as the relationship between space and time. Although it is impossible to directly measure the spatiotemporal features of mental actions, an indirect investigation can be conducted by taking advantage of the constraints existing in planar drawing movements and described by the two-thirds power law (2/3PL). This kinematic law describes one of the most impressive regularities observed in biological movements: movement speed decreases when curvature increases. Here, we compared the duration of identical actual and mental arm movements by changing the constraints imposed by the 2/3PL. In the first two experiments, the length of the trajectory remained constant, while its curvature (Experiment 1) or its number of inflexions (Experiment 2) was manipulated. The results showed that curvature, but not the number of inflexions, proportionally and similarly affected actual and mental movement duration, as expected from the 2/3PL. Two other control experiments confirmed that the results of Experiment 1 were not attributable to eye movements (Experiment 3) or to the perceived length of the displayed trajectory (Experiment 4). Altogether, our findings suggest that mental movement simulation is tuned to the kinematic laws characterizing actions and that kinematics of actual and mental movements is completely specified by the representation of their geometry.     

Cortical mechanisms for trans-saccadic memory and integration of multiple object features

Constructing an internal representation of the world from successive visual fixations, i.e. separated by saccadic eye movements, is known as trans-saccadic perception. Research on trans-saccadic perception (TSP) has been traditionally aimed at resolving the problems of memory capacity and visual integration across saccades. In this paper, we review this literature on TSP with a focus on research showing that egocentric measures of the saccadic eye movement can be used to integrate simple object features across saccades, and that the memory capacity for items retained across saccades, like visual working memory, is restricted to about three to four items. We also review recent transcranial magnetic stimulation experiments which suggest that the right parietal eye field and frontal eye fields play a key functional role in spatial updating of objects in TSP. We conclude by speculating on possible cortical mechanisms for governing egocentric spatial updating of multiple objects in TSP.     

Microstimulation of frontal cortex can reorder a remembered spatial sequence

Complex goal-directed behaviors extend over time and thus depend on the ability to serially order memories and assemble compound, temporally coordinated movements. Theories of sequential processing range from simple associative chaining to hierarchical models in which order is encoded explicitly and separately from sequence components. To examine how short-term memory and planning for sequences might be coded, we used microstimulation to perturb neural activity in the supplementary eye field (SEF) while animals held a sequence of two cued locations in memory over a short delay. We found that stimulation affected the order in which animals saccaded to the locations, but not the memory for which locations were cued. These results imply that memory for sequential order can be dissociated from that of its components. Furthermore, stimulation of the SEF appeared to bias sequence endpoints to converge toward a location in contralateral space, suggesting that this area encodes sequences in terms of their endpoints rather than their individual components.     

Examining Event-Related Potential (ERP) Correlates of Decision Bias in Recognition Memory Judgments

Memory judgments can be based on accurate memory information or on decision bias (the tendency to report that an event is part of episodic memory when one is in fact unsure). Event related potentials (ERP) correlates are important research tools for elucidating the dynamics underlying memory judgments but so far have been established only for investigations of accurate old/new discrimination. To identify the ERP correlates of bias, and observe how these interact with ERP correlates of memory, we conducted three experiments that manipulated decision bias within participants via instructions during recognition memory tests while their ERPs were recorded. In Experiment 1, the bias manipulation was performed between blocks of trials (automatized bias) and compared to trial-by-trial shifts of bias in accord with an external cue (flexibly controlled bias). In Experiment 2, the bias manipulation was performed at two different levels of accurate old/new discrimination as the memory strength of old (studied) items was varied. In Experiment 3, the bias manipulation was added to another, bottom-up driven manipulation of bias induced via familiarity. In the first two Experiments, and in the low familiarity condition of Experiment 3, we found evidence of an early frontocentral ERP component at 320 ms poststimulus (the FN320) that was sensitive to the manipulation of bias via instruction, with more negative amplitudes indexing more liberal bias. By contrast, later during the trial (500–700 ms poststimulus), bias effects interacted with old/new effects across all three experiments. Results suggest that the decision criterion is typically activated early during recognition memory trials, and is integrated with retrieved memory signals and task-specific processing demands later during the trial. More generally, the findings demonstrate how ERPs can help to specify the dynamics of recognition memory processes under top-down and bottom-up controlled retrieval conditions.     

The Effects of Incidentally Learned Temporal and Spatial Predictability on Response Times and Visual Fixations during Target Detection and Discrimination

Responses are quicker to predictable stimuli than if the time and place of appearance is uncertain. Studies that manipulate target predictability often involve overt cues to speed up response times. However, less is known about whether individuals will exhibit faster response times when target predictability is embedded within the inter-trial relationships. The current research examined the combined effects of spatial and temporal target predictability on reaction time (RT) and allocation of overt attention in a sustained attention task. Participants responded as quickly as possible to stimuli while their RT and eye movements were measured. Target temporal and spatial predictability were manipulated by altering the number of: 1) different time intervals between a response and the next target; and 2) possible spatial locations of the target. The effects of target predictability on target detection (Experiment 1) and target discrimination (Experiment 2) were tested. For both experiments, shorter RTs as target predictability increased across both space and time were found. In addition, the influences of spatial and temporal target predictability on RT and the overt allocation of attention were task dependent; suggesting that effective orienting of attention relies on both spatial and temporal predictability. These results indicate that stimulus predictability can be increased without overt cues and detected purely through inter-trial relationships over the course of repeated stimulus presentations.     

Coordinate transformations for eye and arm movements in the brain

Recent work on the coding of spatial information in the brain has significantly advanced our knowledge of sensory to motor transformations on several fronts. The encoding of information referenced to the retina (eye-centered) but modulated by eye position, called a gain field representation, has proved to be very common throughout parietal and occipital cortex. The use of an eye-centered representation as a working memory of spatial location is problematic if the eyes move during the memory period. Details regarding the manner in which the brain solves this problem are beginning to emerge. Finally, the discovery of eye-centered representations of ongoing or intended arm movements has changed the way we think about the order of operations in the sensory to motor coordinate transformation.     

Vision as oculomotor reward: cognitive contributions to the dynamic control of saccadic eye movements

Humans and other primates are equipped with a foveated visual system. As a consequence, we reorient our fovea to objects and targets in the visual field that are conspicuous or that we consider relevant or worth looking at. These reorientations are achieved by means of saccadic eye movements. Where we saccade to depends on various low-level factors such as a targets’ luminance but also crucially on high-level factors like the expected reward or a targets’ relevance for perception and subsequent behavior. Here, we review recent findings how the control of saccadic eye movements is influenced by higher-level cognitive processes. We first describe the pathways by which cognitive contributions can influence the neural oculomotor circuit. Second, we summarize what saccade parameters reveal about cognitive mechanisms, particularly saccade latencies, saccade kinematics and changes in saccade gain. Finally, we review findings on what renders a saccade target valuable, as reflected in oculomotor behavior. We emphasize that foveal vision of the target after the saccade can constitute an internal reward for the visual system and that this is reflected in oculomotor dynamics that serve to quickly and accurately provide detailed foveal vision of relevant targets in the visual field.     

Saccadic target selection as a function of time

Recent evidence indicates that stimulus-driven and goal-directed control of visual selection operate independently and in different time windows (van Zoest et al., 2004). The present study further investigates how eye movements are affected by stimulus-driven and goal-directed control. Observers were presented with search displays consisting of one target, multiple non-targets and one distractor element. The task of observers was to make a fast eye movement to a target immediately following the offset of a central fixation point, an event that either co-occurred with or soon followed the presentation of the search display. Distractor saliency and target-distractor similarity were independently manipulated. The results demonstrated that the effect of distractor saliency was transient and only present for the fastest eye movements, whereas the effect of target-distractor similarity was sustained and present in all but the fastest eye movements. The results support an independent timing account of visual selection.     

Afferent signals from the extraocular muscles of the pigeon modify the electromyogram of these muscles during the vestibulo-ocular reflex.

There is no general agreement on whether afferent signals from the extraocular muscles play any part in oculomotor control. However, we have previously shown that they modify the responses of cells in the oculomotor control system during the vestibulo-ocular reflex (VOR). If, as we suspect, these signals have an important role in the control of the VOR from moment-to-moment, we should be able to demonstrate similar, functionally significant, modifications at the output of the reflex. We have recorded the electromyographic activity of several extraocular muscles of the right eye during the VOR and while imposing movements on the left eye. We describe how the activity of the muscles, reflected in the electromyogram, is modified in specific ways depending on the parameters of the imposed eye movements. The effects of the extraocular afferent signals on the eye-muscle responses to vestibular drive during the slow phase of the VOR appear to be corrective. Thus the present results provide strong evidence that afferent signals from the extraocular muscles are concerned in the control of the reflex from moment-to-moment, and suggest that the wider question of their role in oculomotor control merits further consideration.     

The Relation between Reading Skills and Eye Movement Patterns in Adolescent Readers: Evidence from a Regular Orthography

Over the past decades, the relation between reading skills and eye movement behavior has been well documented in English-speaking cohorts. As English and German differ substantially with regard to orthographic complexity (i.e. grapheme-phoneme correspondence), we aimed to delineate specific characteristics of how reading speed and reading comprehension interact with eye movements in typically developing German-speaking (Austrian) adolescents. Eye movements of 22 participants (14 females; mean age = 13;6 years;months) were tracked while they were performing three tasks, namely silently reading words, texts, and pseudowords. Their reading skills were determined by means of a standardized German reading speed and reading comprehension assessment (Lesegeschwindigkeits- und -verständnistest für Klassen 6−12). We found that (a) reading skills were associated with various eye movement parameters in each of the three reading tasks; (b) better reading skills were associated with an increased efficiency of eye movements, but were primarily linked to spatial reading parameters, such as the number of fixations per word, the total number of saccades and saccadic amplitudes; (c) reading speed was a more reliable predictor for eye movement parameters than reading comprehension; (d) eye movements were highly correlated across reading tasks, which indicates consistent reading performances. Contrary to findings in English-speaking cohorts, the reading skills neither consistently correlated with temporal eye movement parameters nor with the number or percentage of regressions made while performing any of the three reading tasks. These results indicate that, although reading skills are associated with eye movement patterns irrespective of language, the temporal and spatial characteristics of this association may vary with orthographic consistency.     

Activity in the lateral prefrontal cortex reflects multiple steps of future events in action plans

To achieve a behavioral goal in a complex environment, we must plan multiple steps of motor behavior. On planning a series of actions, we anticipate future events that will occur as a result of each action and mentally organize the temporal sequence of events. To investigate the involvement of the lateral prefrontal cortex (PFC) in such multistep planning, we examined neuronal activity in the PFC of monkeys performing a maze task that required the planning of stepwise cursor movements to reach a goal. During the preparatory period, PFC neurons reflected each of all forthcoming cursor movements, rather than arm movements. In contrast, in the primary motor cortex, most neuronal activity reflected arm movements but little of cursor movements during the preparatory period, as well as during movement execution. Our data suggest that the PFC is involved primarily in planning multiple future events that occur as a consequence of behavioral actions.     

Video-oculography in Mice

Eye movements are very important in order to track an object or to stabilize an image on the retina during movement. Animals without a fovea, such as the mouse, have a limited capacity to lock their eyes onto a target. In contrast to these target directed eye movements, compensatory ocular eye movements are easily elicited in afoveate animals^1,2,3,4. Compensatory ocular movements are generated by processing vestibular and optokinetic information into a command signal that will drive the eye muscles. The processing of the vestibular and optokinetic information can be investigated separately and together, allowing the specification of a deficit in the oculomotor system. The oculomotor system can be tested by evoking an optokinetic reflex (OKR), vestibulo-ocular reflex (VOR) or a visually-enhanced vestibulo-ocular reflex (VVOR). The OKR is a reflex movement that compensates for "full-field" image movements on the retina, whereas the VOR is a reflex eye movement that compensates head movements. The VVOR is a reflex eye movement that uses both vestibular as well as optokinetic information to make the appropriate compensation. The cerebellum monitors and is able to adjust these compensatory eye movements. Therefore, oculography is a very powerful tool to investigate brain-behavior relationship under normal as well as under pathological conditions (f.e. of vestibular, ocular and/or cerebellar origin).Testing the oculomotor system, as a behavioral paradigm, is interesting for several reasons. First, the oculomotor system is a well understood neural system⁵. Second, the oculomotor system is relative simple⁶; the amount of possible eye movement is limited by its ball-in-socket architecture ("single joint") and the three pairs of extra-ocular muscles⁷. Third, the behavioral output and sensory input can easily be measured, which makes this a highly accessible system for quantitative analysis⁸. Many behavioral tests lack this high level of quantitative power. And finally, both performance as well as plasticity of the oculomotor system can be tested, allowing research on learning and memory processes⁹.Genetically modified mice are nowadays widely available and they form an important source for the exploration of brain functions at various levels¹⁰. In addition, they can be used as models to mimic human diseases. Applying oculography on normal, pharmacologically-treated or genetically modified mice is a powerful research tool to explore the underlying physiology of motor behaviors under normal and pathological conditions. Here, we describe how to measure video-oculography in mice⁸.     

Updating visual space during motion in depth

Whether we are riding in a car or walking, our internal map of the environment must be continuously updated to maintain spatial constancy. Using a memory eye movement task, we examined whether nonhuman primates can keep track of changes in the distance of nearby objects when moved toward or away from them. We report that memory-guided eye movements take into account the change in distance traveled, illustrating that monkeys can update retinal disparity information in order to reconstruct three-dimensional visual space during motion in depth. This ability was compromised after destruction of the vestibular labyrinths, suggesting that the extraretinal signals needed for updating can arise from vestibular information signaling self-motion through space.     

Distinct control of initiation and metrics of memory-guided saccades and vergence by the FEF: a TMS study

Background

The initiation of memory guided saccades is known to be controlled by the frontal eye field (FEF). Recent physiological studies showed the existence of an area close to FEF that controls also vergence initiation and execution. This study is to explore the effect of transcranial magnetic simulation (TMS) over FEF on the control of memory-guided saccade-vergence eye movements.

Methodology/Principal Findings

Subjects had to make an eye movement in dark towards a target flashed 1 sec earlier (memory delay); the location of the target relative to fixation point was such as to require either a vergence along the median plane, or a saccade, or a saccade with vergence; trials were interleaved. Single pulse TMS was applied on the left or right FEF; it was delivered at 100 ms after the end of memory delay, i.e. extinction of fixation LED that was the “go” signal. Twelve healthy subjects participated in the study. TMS of left or right FEF prolonged the latency of all types of eye movements; the increase varied from 21 to 56 ms and was particularly strong for the divergence movements. This indicates that FEF is involved in the initiation of all types of memory guided movement in the 3D space. TMS of the FEF also altered the accuracy but only for leftward saccades combined with either convergence or divergence; intrasaccadic vergence also increased after TMS of the FEF.

Conclusions/Significance

The results suggest anisotropy in the quality of space memory and are discussed in the context of other known perceptual motor anisotropies.