Spatial updating in human parietal cortex

Single neurons in monkey parietal cortex update visual information in conjunction with eye movements. This remapping of stimulus representations is thought to contribute to spatial constancy. We hypothesized that a similar process occurs in human parietal cortex and that we could visualize it with functional MRI. We scanned subjects during a task that involved remapping of visual signals across hemifields. We observed an initial response in the hemisphere contralateral to the visual stimulus, followed by a remapped response in the hemisphere ipsilateral to the stimulus. We ruled out the possibility that this remapped response resulted from either eye movements or visual stimuli alone. Our results demonstrate that updating of visual information occurs in human parietal cortex.     

Perisaccadic Remapping and Rescaling of Visual Responses in Macaque Superior Colliculus

Visual neurons have spatial receptive fields that encode the positions of objects relative to the fovea. Because foveate animals execute frequent saccadic eye movements, this position information is constantly changing, even though the visual world is generally stationary. Interestingly, visual receptive fields in many brain regions have been found to exhibit changes in strength, size, or position around the time of each saccade, and these changes have often been suggested to be involved in the maintenance of perceptual stability. Crucial to the circuitry underlying perisaccadic changes in visual receptive fields is the superior colliculus (SC), a brainstem structure responsible for integrating visual and oculomotor signals. In this work we have studied the time-course of receptive field changes in the SC. We find that the distribution of the latencies of SC responses to stimuli placed outside the fixation receptive field is bimodal: The first mode is comprised of early responses that are temporally locked to the onset of the visual probe stimulus and stronger for probes placed closer to the classical receptive field. We suggest that such responses are therefore consistent with a perisaccadic rescaling, or enhancement, of weak visual responses within a fixed spatial receptive field. The second mode is more similar to the remapping that has been reported in the cortex, as responses are time-locked to saccade onset and stronger for stimuli placed in the postsaccadic receptive field location. We suggest that these two temporal phases of spatial updating may represent different sources of input to the SC.     

Cortical mechanisms of space-based and object-based attentional control

Visual attention, the mechanism by which observers select relevant or important information from scenes, can be deployed to locations in space or to spatially invariant object representations. Studies have examined both the modulatory effects of attention on the strength of extrastriate cortical representations, and the control of attention by parietal and frontal cortical circuits. Subregions of parietal and frontal cortex are transiently active when attention is voluntarily shifted between spatial locations or object representations. This transient activity may reflect an abrupt shift in the attentional set of the observer, complementing sustained signals that are thought to maintain a given attentive state.     

Visual stability and the motion aftereffect: a psychophysical study revealing spatial updating

Eye movements create an ever-changing image of the world on the retina. In particular, frequent saccades call for a compensatory mechanism to transform the changing visual information into a stable percept. To this end, the brain presumably uses internal copies of motor commands. Electrophysiological recordings of visual neurons in the primate lateral intraparietal cortex, the frontal eye fields, and the superior colliculus suggest that the receptive fields (RFs) of special neurons shift towards their post-saccadic positions before the onset of a saccade. However, the perceptual consequences of these shifts remain controversial. We wanted to test in humans whether a remapping of motion adaptation occurs in visual perception.The motion aftereffect (MAE) occurs after viewing of a moving stimulus as an apparent movement to the opposite direction. We designed a saccade paradigm suitable for revealing pre-saccadic remapping of the MAE. Indeed, a transfer of motion adaptation from pre-saccadic to post-saccadic position could be observed when subjects prepared saccades. In the remapping condition, the strength of the MAE was comparable to the effect measured in a control condition (33±7% vs. 27±4%). Contrary, after a saccade or without saccade planning, the MAE was weak or absent when adaptation and test stimulus were located at different retinal locations, i.e. the effect was clearly retinotopic. Regarding visual cognition, our study reveals for the first time predictive remapping of the MAE but no spatiotopic transfer across saccades. Since the cortical sites involved in motion adaptation in primates are most likely the primary visual cortex and the middle temporal area (MT/V5) corresponding to human MT, our results suggest that pre-saccadic remapping extends to these areas, which have been associated with strict retinotopy and therefore with classical RF organization. The pre-saccadic transfer of visual features demonstrated here may be a crucial determinant for a stable percept despite saccades.     

Right-hemispheric dominance for visual remapping in humans

We review evidence showing a right-hemispheric dominance for visuo-spatial processing and representation in humans. Accordingly, visual disorganization symptoms (intuitively related to remapping impairments) are observed in both neglect and constructional apraxia. More specifically, we review findings from the intervening saccade paradigm in humans--and present additional original data--which suggest a specific role of the asymmetrical network at the temporo-parietal junction (TPJ) in the right hemisphere in visual remapping: following damage to the right dorsal posterior parietal cortex (PPC) as well as part of the corpus callosum connecting the PPC to the frontal lobes, patient OK in a double-step saccadic task exhibited an impairment when the second saccade had to be directed rightward. This singular and lateralized deficit cannot result solely from the patient's cortical lesion and, therefore, we propose that it is due to his callosal lesion that may specifically interrupt the interhemispheric transfer of information necessary to execute accurate rightward saccades towards a remapped target location. This suggests a specialized right-hemispheric network for visuo-spatial remapping that subsequently transfers target location information to downstream planning regions, which are symmetrically organized.     

Goal-related activity in V4 during free viewing visual search. Evidence for a ventral stream visual salience map

Natural exploration of complex visual scenes depends on saccadic eye movements toward important locations. Saccade targeting is thought to be mediated by a retinotopic map that represents the locations of salient features. In this report, we demonstrate that extrastriate ventral area V4 contains a retinotopic salience map that guides exploratory eye movements during a naturalistic free viewing visual search task. In more than half of recorded cells, visually driven activity is enhanced prior to saccades that move the fovea toward the location previously occupied by a neuron's spatial receptive field. This correlation suggests that bottom-up processing in V4 influences the oculomotor planning process. Half of the neurons also exhibit top-down modulation of visual responses that depends on search target identity but not visual stimulation. Convergence of bottom-up and top-down processing streams in area V4 results in an adaptive, dynamic map of salience that guides oculomotor planning during natural vision.     

Retinotopic organization of striate and extrastriate visual cortex in the golden hamster (Mesocricetus auratus).

The visual topography within striate and lateral extrastriate visual cortices was studied in adult hamsters. The cortical areas 17 and 18a in the left hemisphere were electrophysiologically mapped upon stimulation of the right eye, correlating receptive field positions in the visual field with cortical recording sites. Reference lesions were placed at selected cortical sites. Like in rats and other mammals, the lateral extrastriate cortex contained multiple representations of the visual field. Rostral area 18a contained the rostrolateral maps, with medial and lateral divisions. More caudally and sharing a common border with V1, maps in lateromedial, posterolateral and posterior areas were found. More laterally and forming a "third tier" of visual maps, anterolateral, laterolateral-anterior, laterolateral and laterolateral-posterior areas were found. There was also an indication of a possible pararhinal map. The plan so defined is virtually identical to that of rats. The results may be useful to understand a basic mammalian plan in the organization of the visual cortex.     

Changes in visual receptive fields with microstimulation of frontal cortex

The influence of attention on visual cortical neurons has been described in terms of its effect on the structure of receptive fields (RFs), where multiple stimuli compete to drive neural responses and ultimately behavior. We stimulated the frontal eye field (FEF) of passively fixating monkeys and produced changes in V4 responses similar to known effects of voluntary attention. Subthreshold FEF stimulation enhanced visual responses at particular locations within the RF and altered the interaction between pairs of RF stimuli to favor those aligned with the activated FEF site. Thus, we could influence which stimulus drove the responses of individual V4 neurons. These results suggest that spatial signals involved in saccade preparation are used to covertly select among multiple stimuli appearing within the RFs of visual cortical neurons.     

The role of the posterior parietal cortex in coordinate transformations for visual-motor integration

Lesion to the posterior parietal cortex in monkeys and humans produces spatial deficits in movement and perception. In recording experiments from area 7a, a cortical subdivision in the posterior parietal cortex in monkeys, we have found neurons whose responses are a function of both the retinal location of visual stimuli and the position of the eyes in the orbits. By combining these signals area 7 a neurons code the location of visual stimuli with respect to the head. However, these cells respond over only limited ranges of eye positions (eye-position-dependent coding). To code location in craniotopic space at all eye positions (eye-position-independent coding) an additional step in neural processing is required that uses information distributed across populations of area 7a neurons. We describe here a neural network model, based on back-propagation learning, that both demonstrates how spatial location could be derived from the population response of area 7a neurons and accurately accounts for the observed response properties of these neurons.     

Visual attention mediated by biased competition in extrastriate visual cortex.

According to conventional neurobiological accounts of visual attention, attention serves to enhance extrastriate neuronal responses to a stimulus at one spatial location in the visual field. However, recent results from recordings in extrastriate cortex of monkeys suggest that any enhancing effect of attention is best understood in the context of competitive interactions among neurons representing all of the stimuli present in the visual field. These interactions can be biased in favour of behaviourally relevant stimuli as a result of many different processes, both spatial and non-spatial, and both bottom-up and top-down. The resolution of this competition results in the suppression of the neuronal representations of behaviourally irrelevant stimuli in extrastriate cortex. A main source of top-down influence may derive from neuronal systems underlying working memory.     

A competitive integration model of exogenous and endogenous eye movements

We present a model of the eye movement system in which the programming of an eye movement is the result of the competitive integration of information in the superior colliculi (SC). This brain area receives input from occipital cortex, the frontal eye fields, and the dorsolateral prefrontal cortex, on the basis of which it computes the location of the next saccadic target. Two critical assumptions in the model are that cortical inputs are not only excitatory, but can also inhibit saccades to specific locations, and that the SC continue to influence the trajectory of a saccade while it is being executed. With these assumptions, we account for many neurophysiological and behavioral findings from eye movement research. Interactions within the saccade map are shown to account for effects of distractors on saccadic reaction time (SRT) and saccade trajectory, including the global effect and oculomotor capture. In addition, the model accounts for express saccades, the gap effect, saccadic reaction times for antisaccades, and recorded responses from neurons in the SC and frontal eye fields in these tasks.     

Multimodal integration for the representation of space in the posterior parietal cortex.

The posterior parietal cortex has long been considered an ''association'' area that combines information from different sensory modalities to form a cognitive representation of space. However, until recently little has been known about the neural mechanisms responsible for this important cognitive process. Recent experiments from the author''s laboratory indicate that visual, somatosensory, auditory and vestibular signals are combined in areas LIP and 7a of the posterior parietal cortex. The integration of these signals can represent the locations of stimuli with respect to the observer and within the environment. Area MSTd combines visual motion signals, similar to those generated during an observer''s movement through the environment, with eye-movement and vestibular signals. This integration appears to play a role in specifying the path on which the observer is moving. All three cortical areas combine different modalities into common spatial frames by using a gain-field mechanism. The spatial representations in areas LIP and 7a appear to be important for specifying the locations of targets for actions such as eye movements or reaching; the spatial representation within area MSTd appears to be important for navigation and the perceptual stability of motion signals.     

Effect of expected reward magnitude on the response of neurons in the dorsolateral prefrontal cortex of the macaque

The dorsolateral prefrontal cortex plays a critical role in guiding actions that ensue seconds after an instruction. We recorded from neurons in area 46 and the frontal eye field (FEF) while monkeys performed a memory-guided eye movement task. A visual cue signaled whether a small or large liquid reward would accompany a correct response. Many neurons in area 46 responded more when the monkey expected a larger reward. Reward-related enhancement was evident throughout the memory period and was most pronounced when the remembered target appeared in the neuron's response field. Enhancement was not present in the FEF. The mixture of neural signals representing spatial working memory and reward expectation appears to be a distinct feature of area 46.     

Decoding successive computational stages of saliency processing

An important requirement for vision is to identify interesting and relevant regions of the environment for further processing. Some models assume that salient locations from a visual scene are encoded in a dedicated spatial saliency map [1, 2]. Then, a winner-take-all (WTA) mechanism [1, 2] is often believed to threshold the graded saliency representation and identify the most salient position in the visual field. Here we aimed to assess whether neural representations of graded saliency and the subsequent WTA mechanism can be dissociated. We presented images of natural scenes while subjects were in a scanner performing a demanding fixation task, and thus their attention was directed away. Signals in early visual cortex and posterior intraparietal sulcus (IPS) correlated with graded saliency as defined by a computational saliency model. Multivariate pattern classification [3, 4] revealed that the most salient position in the visual field was encoded in anterior IPS and frontal eye fields (FEF), thus reflecting a potential WTA stage. Our results thus confirm that graded saliency and WTA-thresholded saliency are encoded in distinct neural structures. This could provide the neural representation required for rapid and automatic orientation toward salient events in natural environments.     

Multiple spotlights of attentional selection in human visual cortex

Spatially directed attention strongly enhances visual perceptual processing. The metaphor of the "spotlight" has long been used to describe spatial attention; however, there has been considerable debate as to whether spatial attention must be unitary or may be split between discrete regions of space. This question was addressed here through functional MR imaging of human subjects as they performed a task that required simultaneous attention to two briefly displayed and masked targets at locations separated by distractor stimuli. These data reveal retinotopically specific enhanced activation in striate and extrastriate visual cortical representations of the two attended stimuli and no enhancement at the intervening representation of distractor stimuli. This finding of two spotlights was obtained within a single cortical hemisphere and across the two hemispheres. This provides direct evidence that spatial attention can select, in parallel, multiple low-level perceptual representations.     

Reward-dependent gain and bias of visual responses in primate superior colliculus

Eye movements are often influenced by expectation of reward. Using a memory-guided saccade task with an asymmetric reward schedule, we show that visual responses of monkey SC neurons increase when the visual stimulus indicates an upcoming reward. The increase occurred in two distinct manners: (1) reactively, as an increase in the gain of the visual response when the stimulus indicated an upcoming reward; (2) proactively, as an increase in anticipatory activity when reward was expected in the neuron's response field. These effects were observed mostly in saccade-related SC neurons in the deeper layer which would receive inputs from the cortical eye fields and the basal ganglia. These results, together with recent findings, suggest that the gain modulation may be determined by the inputs from both the cortical eye fields and the basal ganglia, whereas the anticipatory bias may be derived mainly from the basal ganglia.     

Axonal Dynamics of Excitatory and Inhibitory Neurons in Somatosensory Cortex

Cortical topography can be remapped as a consequence of sensory deprivation, suggesting that cortical circuits are continually modified by experience. To see the effect of altered sensory experience on specific components of cortical circuits, we imaged neurons, labeled with a genetically modified adeno-associated virus, in the intact mouse somatosensory cortex before and after whisker plucking. Following whisker plucking we observed massive and rapid reorganization of the axons of both excitatory and inhibitory neurons, accompanied by a transient increase in bouton density. For horizontally projecting axons of excitatory neurons there was a net increase in axonal projections from the non-deprived whisker barrel columns into the deprived barrel columns. The axon collaterals of inhibitory neurons located in the deprived whisker barrel columns retracted in the vicinity of their somata and sprouted long-range projections beyond their normal reach towards the non-deprived whisker barrel columns. These results suggest that alterations in the balance of excitation and inhibition in deprived and non-deprived barrel columns underlie the topographic remapping associated with sensory deprivation.     

Frontal cortical control of smooth-pursuit

To maintain optimal clarity of objects moving slowly in three dimensional space, frontal eyed-primates use both smooth-pursuit and vergence (depth) eye movements to track precisely those objects and maintain their images on the foveae of left and right eyes. The caudal parts of the frontal eye fields contain neurons that discharge during smooth-pursuit. Recent results have provided a new understanding of the roles of the frontal eye field pursuit area and suggest that it may control the gain of pursuit eye movements, code predictive visual signals that drive pursuit, and code commands for smooth eye movements in a three dimensional coordinate frame.     

Towards understanding of the cortical network underlying associative memory

Declarative knowledge and experiences are represented in the association cortex and are recalled by reactivation of the neural representation. Electrophysiological experiments have revealed that associations between semantically linked visual objects are formed in neural representations in the temporal and limbic cortices. Memory traces are created by the reorganization of neural circuits. These regions are reactivated during retrieval and contribute to the contents of a memory. Two different types of retrieval signals are suggested as follows: automatic and active. One flows backward from the medial temporal lobe during the automatic retrieval process, whereas the other is conveyed as a top-down signal from the prefrontal cortex to the temporal cortex during the active retrieval process. By sending the top-down signal, the prefrontal cortex manipulates and organizes to-be-remembered information, devises strategies for retrieval and monitors the outcome. To further understand the neural mechanism of memory, the following two complementary views are needed: how the multiple cortical areas in the brain-wide network interact to orchestrate cognitive functions and how the properties of single neurons and their synaptic connections with neighbouring neurons combine to form local circuits and to exhibit the function of each cortical area. We will discuss some new methodological innovations that tackle these challenges.     

Representation of binocular surfaces by cortical neurons

Useful representations of the three-dimensional (3D) world go beyond assigning depth to individual points, building maps of surfaces and shapes. Studies in a wide range of extrastriate cortical areas have shown that single neurons show selective responses to 3D surfaces. The extent to which this advances the representation beyond that provided by the earliest binocular signals requires careful evaluation. We conclude that current data are not sufficient to identify distinctive contributions from different cortical areas to the binocular representation of 3D surfaces.