The peri-saccadic perception of objects and space

Eye movements affect object localization and object recognition. Around saccade onset, briefly flashed stimuli appear compressed towards the saccade target, receptive fields dynamically change position, and the recognition of objects near the saccade target is improved. These effects have been attributed to different mechanisms. We provide a unifying account of peri-saccadic perception explaining all three phenomena by a quantitative computational approach simulating cortical cell responses on the population level. Contrary to the common view of spatial attention as a spotlight, our model suggests that oculomotor feedback alters the receptive field structure in multiple visual areas at an intermediate level of the cortical hierarchy to dynamically recruit cells for processing a relevant part of the visual field. The compression of visual space occurs at the expense of this locally enhanced processing capacity.     

Changes in visual receptive fields with microstimulation of frontal cortex

The influence of attention on visual cortical neurons has been described in terms of its effect on the structure of receptive fields (RFs), where multiple stimuli compete to drive neural responses and ultimately behavior. We stimulated the frontal eye field (FEF) of passively fixating monkeys and produced changes in V4 responses similar to known effects of voluntary attention. Subthreshold FEF stimulation enhanced visual responses at particular locations within the RF and altered the interaction between pairs of RF stimuli to favor those aligned with the activated FEF site. Thus, we could influence which stimulus drove the responses of individual V4 neurons. These results suggest that spatial signals involved in saccade preparation are used to covertly select among multiple stimuli appearing within the RFs of visual cortical neurons.     

Eye movements modulate visual receptive fields of V4 neurons

The receptive field, defined as the spatiotemporal selectivity of neurons to sensory stimuli, is central to our understanding of the neuronal mechanisms of perception. However, despite the fact that eye movements are critical during normal vision, the influence of eye movements on the structure of receptive fields has never been characterized. Here, we map the receptive fields of macaque area V4 neurons during saccadic eye movements and find that receptive fields are remarkably dynamic. Specifically, before the initiation of a saccadic eye movement, receptive fields shrink and shift towards the saccade target. These spatiotemporal dynamics may enhance information processing of relevant stimuli during the scanning of a visual scene, thereby assisting the selection of saccade targets and accelerating the analysis of the visual scene during free viewing.     

A model of the saccade-generating system that accounts for trajectory variations produced by competing visual stimuli

Variable saccade trajectories are produced in visual search paradigms in which multiple potential target stimuli are present. These variable trajectories provide a rich source of information that may lead to a deeper understanding of the basic control mechanisms of the saccadic system. We have used published behavioral observations and neural recordings in the superior colliculus (SC), gathered in monkeys performing visual search paradigms, to guide the construction of a new distributed model of the saccadic system. The new model can account for many of the variations in saccade trajectory produced by the appearance of multiple visual stimuli in a search paradigm. The model uses distributed feedback about current eye motion from the brainstem to the SC to reduce activity there at physiologically realistic rates during saccades. The long-range lateral inhibitory connections between SC cells used in previous models have been eliminated to match recent physiological evidence. The model features interactions between visually activated multiple populations of cells in the SC and distributed and topologically organized inhibitory input to the SC from the SNr to produce some of the types of variable saccadic trajectories, including slightly curved and averaging saccades, observed in visual search tasks. The distributed perisaccadic disinhibition of SC from the substantia nigra (SNr) is assumed to have broad spatial tuning. In order to produce the strongly curved saccades occasionally recorded in visual search, the existence of a parallel input to the saccadic burst generators in addition to that provided by the distributed input from the SC is required. The spatiotemporal form of this additional parallel input is computed based on the assumption that the input from the model SC is realistic. In accordance with other recent models, it is assumed that the parallel input comes from the cerebellum, but our model predicts that the parallel input is delayed during highly curved saccadic trajectories.     

Neurons with visual receptive fields independent of eye position in the caudal section of the ventral bank of cat cruciate slucus

Neurons responding to tactile and visual stimulation were found in the caudal section of the cruciate slucus ventral bank in awake cats. Tactile receptive fields were located on the face, mainly around the mouth. Visual stimuli evoked a response when presented close to the tactile receptive field. It was found that the visual responses of these bimodal neurons located in layer VI of the cortex display spatial consistency. The position of these visual receptive fields remained constant through saccadic eye movements, while still linked to the tactile receptive field.Institute for Research into Information Transmission, Academy of Sciences of the USSR, Moscow. Translated from Neifofiziologiya, Vol. 18, No. 6, pp. 800–805, November–December, 1986.     

Visual receptive field organization

Increasingly systematic approaches to quantifying receptive fields in primary visual cortex, combined with inspired ideas about functional circuitry, non-linearities, and visual stimuli, are bringing new interest to classical problems. This includes the distinction and hierarchy between simple and complex cells, the mechanisms underlying the receptive field surround, and debates about optimal stimuli for mapping receptive fields. An important new problem arises from recent observations of stimulus-dependent spatial and temporal summation in primary visual cortex. It appears that the receptive field can no longer be considered unique, and we might have to relinquish this cherished notion as the embodiment of neuronal function in primary visual cortex.     

Features and functions of nonlinear spatial integration by retinal ganglion cells

Ganglion cells in the vertebrate retina integrate visual information over their receptive fields. They do so by pooling presynaptic excitatory inputs from typically many bipolar cells, which themselves collect inputs from several photoreceptors. In addition, inhibitory interactions mediated by horizontal cells and amacrine cells modulate the structure of the receptive field. In many models, this spatial integration is assumed to occur in a linear fashion. Yet, it has long been known that spatial integration by retinal ganglion cells also incurs nonlinear phenomena. Moreover, several recent examples have shown that nonlinear spatial integration is tightly connected to specific visual functions performed by different types of retinal ganglion cells. This work discusses these advances in understanding the role of nonlinear spatial integration and reviews recent efforts to quantitatively study the nature and mechanisms underlying spatial nonlinearities. These new insights point towards a critical role of nonlinearities within ganglion cell receptive fields for capturing responses of the cells to natural and behaviorally relevant visual stimuli. In the long run, nonlinear phenomena of spatial integration may also prove important for implementing the actual neural code of retinal neurons when designing visual prostheses for the eye.     

Parietal neurons encoding visual space in a head-frame of reference.

Neurons of the visual system are known to have receptive fields organized in retinotopic coordinates. We wanted to test whether visual neurons existed whose receptive fields were organized in spatial coordinates. Extracellular recordings from single cells were carried out in one area of the posterior parietal cortex (area V6) of a behaving macaque monkey. Among a great majority of retinotopically organized visual cells, neurons whose visual receptive field did not shift with gaze were also found. These cells responded to the visual stimulation of the same spatial position independently of the animal's direction of gaze, that is, their receptive field was anchored to an absolute spatial location. We suggest that these neurons directly encode visual space and are involved in programming visually-guided motor actions in space.     

Adaptation to stimulus contrast and correlations during natural visual stimulation

In this study, we characterize the adaptation of neurons in the cat lateral geniculate nucleus to changes in stimulus contrast and correlations. By comparing responses to high- and low-contrast natural scene movie and white noise stimuli, we show that an increase in contrast or correlations results in receptive fields with faster temporal dynamics and stronger antagonistic surrounds, as well as decreases in gain and selectivity. We also observe contrast- and correlation-induced changes in the reliability and sparseness of neural responses. We find that reliability is determined primarily by processing in the receptive field (the effective contrast of the stimulus), while sparseness is determined by the interactions between several functional properties. These results reveal a number of adaptive phenomena and suggest that adaptation to stimulus contrast and correlations may play an important role in visual coding in a dynamic natural environment.     

Building better models of visual cortical receptive fields

Scientists usually study the receptive fields of visual cortical neurons by measuring responses to "optimal stimuli." In this issue of Neuron, Rust and colleagues have taken a promising alternative approach: build a receptive field model based on the cell responses to a stimulus subset and then use the model to predict responses to other stimuli.     

A computational theory of visual receptive fields

A receptive field constitutes a region in the visual field where a visual cell or a visual operator responds to visual stimuli. This paper presents a theory for what types of receptive field profiles can be regarded as natural for an idealized vision system, given a set of structural requirements on the first stages of visual processing that reflect symmetry properties of the surrounding world. These symmetry properties include (i) covariance properties under scale changes, affine image deformations, and Galilean transformations of space–time as occur for real-world image data as well as specific requirements of (ii) temporal causality implying that the future cannot be accessed and (iii) a time-recursive updating mechanism of a limited temporal buffer of the past as is necessary for a genuine real-time system. Fundamental structural requirements are also imposed to ensure (iv) mutual consistency and a proper handling of internal representations at different spatial and temporal scales. It is shown how a set of families of idealized receptive field profiles can be derived by necessity regarding spatial, spatio-chromatic, and spatio-temporal receptive fields in terms of Gaussian kernels, Gaussian derivatives, or closely related operators. Such image filters have been successfully used as a basis for expressing a large number of visual operations in computer vision, regarding feature detection, feature classification, motion estimation, object recognition, spatio-temporal recognition, and shape estimation. Hence, the associated so-called scale-space theory constitutes a both theoretically well-founded and general framework for expressing visual operations. There are very close similarities between receptive field profiles predicted from this scale-space theory and receptive field profiles found by cell recordings in biological vision. Among the family of receptive field profiles derived by necessity from the assumptions, idealized models with very good qualitative agreement are obtained for (i) spatial on-center/off-surround and off-center/on-surround receptive fields in the fovea and the LGN, (ii) simple cells with spatial directional preference in V1, (iii) spatio-chromatic double-opponent neurons in V1, (iv) space–time separable spatio-temporal receptive fields in the LGN and V1, and (v) non-separable space–time tilted receptive fields in V1, all within the same unified theory. In addition, the paper presents a more general framework for relating and interpreting these receptive fields conceptually and possibly predicting new receptive field profiles as well as for pre-wiring covariance under scaling, affine, and Galilean transformations into the representations of visual stimuli. This paper describes the basic structure of the necessity results concerning receptive field profiles regarding the mathematical foundation of the theory and outlines how the proposed theory could be used in further studies and modelling of biological vision. It is also shown how receptive field responses can be interpreted physically, as the superposition of relative variations of surface structure and illumination variations, given a logarithmic brightness scale, and how receptive field measurements will be invariant under multiplicative illumination variations and exposure control mechanisms.     

Prediction of orientation selectivity from receptive field architecture in simple cells of cat visual cortex

From the intracellularly recorded responses to small, rapidly flashed spots, we have quantitatively mapped the receptive fields of simple cells in the cat visual cortex. We then applied these maps to a feedforward model of orientation selectivity. Both the preferred orientation and the width of orientation tuning of the responses to oriented stimuli were well predicted by the model. Where tested, the tuning curve was well predicted at different spatial frequencies. The model was also successful in predicting certain features of the spatial frequency selectivity of the cells. It did not successfully predict the amplitude of the responses to drifting gratings. Our results show that the spatial organization of the receptive field can account for a large fraction of the orientation selectivity of simple cells.     

Spectral characteristics of stimuli and responses in a complex receptive field]

Responses of a complex receptive field of visual cortex of the cat are determined by the energy of stimuli in the band pass of equivalent space frequency characteristic of the receptive field. This is a proof that the complex receptive fields are spatial frequency band pass filters.     

Sensory and behavioral properties of neurons in posterior parietal cortex of the awake, trained monkey.

Neurons in posterior parietal cortex of the awake, trained monkey respond to passive visual and/or somatosensory stimuli. In general, the receptive fields of these cells are large and nonspecific. When these neurons are studied during visually guided hand movements and eye movements, most of their activity can be accounted for by passive sensory stimulation. However, for some visual cells, the response to a stimulus is enhanced when it is to be the target for a saccadic eye movement. This enhancement is selective for eye movements into the visual receptive field since it does not occur with eye movements to other parts of the visual field. Cells that discharge in association with a visual fixation task have foveal receptive fields and respond to the spots of light used as fixation targets. Cells discharging selectively in association with different directions of tracking eye movements have directionally selective responses to moving visual stimuli. Every cell in our sample discharging in association with movement could be driven by passive sensory stimuli. We conclude that the activity of neurons in posterior parietal cortex is dependent on and indicative of external stimuli but not predictive of movement.     

Saccade generation by the frontal eye fields in rhesus monkeys is separable from visual detection and bottom-up attention shift

The frontal eye fields (FEF), originally identified as an oculomotor cortex, have also been implicated in perceptual functions, such as constructing a visual saliency map and shifting visual attention. Further dissecting the area's role in the transformation from visual input to oculomotor command has been difficult because of spatial confounding between stimuli and responses and consequently between intermediate cognitive processes, such as attention shift and saccade preparation. Here we developed two tasks in which the visual stimulus and the saccade response were dissociated in space (the extended memory-guided saccade task), and bottom-up attention shift and saccade target selection were independent (the four-alternative delayed saccade task). Reversible inactivation of the FEF in rhesus monkeys disrupted, as expected, contralateral memory-guided saccades, but visual detection was demonstrated to be intact at the same field. Moreover, saccade behavior was impaired when a bottom-up shift of attention was not a prerequisite for saccade target selection, indicating that the inactivation effect was independent of the previously reported dysfunctions in bottom-up attention control. These findings underscore the motor aspect of the area's functions, especially in situations where saccades are generated by internal cognitive processes, including visual short-term memory and long-term associative memory.     

Motion sensitive interneurons in the optomotor system of the fly

The functional properties of the three horizontal cells (north horizontal cell, HSN; equatorial horizontal cell, HSE; south horizontal cell, HSS) in the lobula plate of the blowflyCalliphora erythrocephala were investigated electrophysiologically. 1. The receptive fields of the HSN, HSE, and HSS cover the dorsal, equatorial and ventral part of the ipsilateral visual field, respectively. In all three cells, the sensitivity to visual stimulation is highest in the frontal visual field and decreases laterally. The receptive fields and spatial sensitivity distributions of the horizontal cells are directly determined by the position and extension of their dendritic fields in the lobula plate and the dendritic density distributions within these fields. 2. The horizontal cells respond mainly to progressive (front to back) motion and are inhibited by motion in the reverse direction, the preferred and null direction being antiparallel. The amplitudes of motion induced excitatory and inhibitory responses decline like a cosine function with increasing deviation of the direction of motion from the preferred direction. Stimulation with motion in directions perpendicular to the preferred direction is ineffective. 3. The preferred directions of the horizontal cells show characteristic gradual orientation changes in different parts of the receptive fields: they are horizontally oriented only in the equatorial region and increasingly tilted vertically towards the dorsofrontal and ventrofrontal margins of the visual field. These orientation changes can be correlated with equivalent changes in the local orientation of the lattice of ommatidial axes in the pertinent compound eye. 4. The response amplitudes of the horizontal cells under stimulation with a moving periodic grating depend strongly on the contrast frequency of the stimulus. Maximal responses were found at contrast frequencies of 2–5 Hz. 5. The spatial integration properties of the horizontal cells (studied in the HSE) are highly nonlinear. Under stimulation with extended moving patterns, their response amplitudes are nearly independent of the size of the stimuli. It is demonstrated that this response behaviour does not result from postsynaptic saturation in the dendrites of the cells. The results indicate that the horizontal system is essentially involved in the neural control of optomotor torque responses performed by the fly in order to minimize unvoluntary deviations from a straight flight course.     

Adaptive surround modulation in cortical area MT

Visual motion perception relies on two opposing operations: integration and segmentation. Integration overcomes motion ambiguity in the visual image by spatial pooling of motion signals, whereas segmentation identifies differences between adjacent moving objects. For visual motion area MT, previous investigations have reported that stimuli in the receptive field surround, which do not elicit a response when presented alone, can nevertheless modulate responses to stimuli in the receptive field center. The directional tuning of this "surround modulation" has been found to be mainly antagonistic and hence consistent with segmentation. Here, we report that surround modulation in area MT can be either antagonistic or integrative depending upon the visual stimulus. Both types of modulation were delayed relative to response onset. Our results suggest that the dominance of antagonistic modulation in previous MT studies was due to stimulus choice and that segmentation and integration are achieved, in part, via adaptive surround modulation.     

Contextual modulation of V1 receptive fields depends on their spatial symmetry

The apparent receptive field characteristics of sensory neurons depend on the statistics of the stimulus ensemble—a nonlinear phenomenon often called contextual modulation. Since visual cortical receptive fields determined from simple stimuli typically do not predict responses to complex stimuli, understanding contextual modulation is crucial to understanding responses to natural scenes. To analyze contextual modulation, we examined how apparent receptive fields differ for two stimulus ensembles that are matched in first- and second-order statistics, but differ in their feature content: one ensemble is enriched in elongated contours. To identify systematic trends across the neural population, we used a multidimensional scaling method, the Procrustes transformation. We found that contextual modulation of receptive field components increases with their spatial extent. More surprisingly, we also found that odd-symmetric components change systematically, but even-symmetric components do not. This symmetry dependence suggests that contextual modulation is driven by oriented On/Off dyads, i.e., modulation of the strength of intracortically-generated signals. Electronic supplementary material  The online version of this article (doi:) contains supplementary material, which is available to authorized users.     

Response histogram shapes and tuning curves: The predicted responses of several cortical cell types to drifting gratings stimuli

The responses to visual stimuli of simple cortical cells show linear spatial summation within and between their receptive field subunits. Complex cortical cells do not show this linearity. We analyzed the simulated responses to drifting sinusoidal grating stimuli of simple and of several types of complex cells. The complex cells, whose responses are seen to be half-wave rectified before pooling, have receptive fields consisting of two or more DOG (difference-of-Gaussians) shaped subunits. In both cases of stimulation by contrast-reversal gratings or drifting gratings, the cells' response as a function of spatial frequency is affected by the subunit distances 2 and the stimulation frequency . Furthermore, an increased number of subunits (a larger receptive field) yields a narrower peak tuning curve with decreased modulation depth for many of the spatial frequencies. The average and the peak response tuning curves are compared for the different receptive field types.