Eye movements modulate visual receptive fields of V4 neurons

The receptive field, defined as the spatiotemporal selectivity of neurons to sensory stimuli, is central to our understanding of the neuronal mechanisms of perception. However, despite the fact that eye movements are critical during normal vision, the influence of eye movements on the structure of receptive fields has never been characterized. Here, we map the receptive fields of macaque area V4 neurons during saccadic eye movements and find that receptive fields are remarkably dynamic. Specifically, before the initiation of a saccadic eye movement, receptive fields shrink and shift towards the saccade target. These spatiotemporal dynamics may enhance information processing of relevant stimuli during the scanning of a visual scene, thereby assisting the selection of saccade targets and accelerating the analysis of the visual scene during free viewing.     

Changes in visual receptive fields with microstimulation of frontal cortex

The influence of attention on visual cortical neurons has been described in terms of its effect on the structure of receptive fields (RFs), where multiple stimuli compete to drive neural responses and ultimately behavior. We stimulated the frontal eye field (FEF) of passively fixating monkeys and produced changes in V4 responses similar to known effects of voluntary attention. Subthreshold FEF stimulation enhanced visual responses at particular locations within the RF and altered the interaction between pairs of RF stimuli to favor those aligned with the activated FEF site. Thus, we could influence which stimulus drove the responses of individual V4 neurons. These results suggest that spatial signals involved in saccade preparation are used to covertly select among multiple stimuli appearing within the RFs of visual cortical neurons.     

Four concurrent feedforward and feedback networks with different roles in the visual cortical hierarchy

Visual stimuli evoke fast-evolving activity patterns that are distributed across multiple cortical areas. These areas are hierarchically structured, as indicated by their anatomical projections, but how large-scale feedforward and feedback streams are functionally organized in this system remains an important missing clue to understanding cortical processing. By analyzing visual evoked responses in laminar recordings from 6 cortical areas in awake mice, we uncovered a dominant feedforward network with scale-free interactions in the time domain. In addition, we established the simultaneous presence of a gamma band feedforward and 2 low frequency feedback networks, each with a distinct laminar functional connectivity profile, frequency spectrum, temporal dynamics, and functional hierarchy. We could identify distinct roles for each of these 4 processing streams, by leveraging stimulus contrast effects, analyzing receptive field (RF) convergency along functional interactions, and determining relationships to spiking activity. Our results support a dynamic dual counterstream view of hierarchical processing and provide new insight into how separate functional streams can simultaneously and dynamically support visual processes.

Visual stimuli evoke fast-evolving activity patterns that are distributed across multiple cortical areas, but how large-scale feedforward and feedback streams are functionally organized in this system remains unclear. Visual evoked responses in laminar recordings from six cortical areas in awake mice reveal how layers and rhythms dynamically orchestrate functional streams in vision.     

A simple translation in cortical log-coordinates may account for the pattern of saccadic localization errors

During saccadic eye movements, the visual world shifts rapidly across the retina. Perceptual continuity is thought to be maintained by active neural mechanisms that compensate for this displacement, bringing the presaccadic scene into a postsaccadic reference frame. Because of this active mechanism, objects appearing briefly around the time of the saccade are perceived at erroneous locations, a phenomenon called perisaccadic mislocalization. The position and direction of localization errors can inform us about the different reference frames involved. It has been found, for example, that errors are not simply made in the direction of the saccade but directed toward the saccade target, indicating that the compensatory mechanism involves spatial compression rather than translation. A recent study confirmed that localization errors also occur in the direction orthogonal to saccade direction, but only for eccentricities far from the fovea, beyond the saccade target. This spatially specific pattern of distortion cannot be explained by a simple compression of space around the saccade target. Here I show that a change of reference frames (i.e., translation) in cortical (logarithmic) coordinates, taking into account the cortical magnification factor, can accurately predict these spatial patterns of mislocalization. The flashed object projects onto the cortex in presaccadic (fovea-centered) coordinates but is perceived in postsaccadic (target-centered) coordinates.     

Invariance of visual operations at the level of receptive fields

The brain is able to maintain a stable perception although the visual stimuli vary substantially on the retina due to geometric transformations and lighting variations in the environment. This paper presents a theory for achieving basic invariance properties already at the level of receptive fields. Specifically, the presented framework comprises (i) local scaling transformations caused by objects of different size and at different distances to the observer, (ii) locally linearized image deformations caused by variations in the viewing direction in relation to the object, (iii) locally linearized relative motions between the object and the observer and (iv) local multiplicative intensity transformations caused by illumination variations. The receptive field model can be derived by necessity from symmetry properties of the environment and leads to predictions about receptive field profiles in good agreement with receptive field profiles measured by cell recordings in mammalian vision. Indeed, the receptive field profiles in the retina, LGN and V1 are close to ideal to what is motivated by the idealized requirements. By complementing receptive field measurements with selection mechanisms over the parameters in the receptive field families, it is shown how true invariance of receptive field responses can be obtained under scaling transformations, affine transformations and Galilean transformations. Thereby, the framework provides a mathematically well-founded and biologically plausible model for how basic invariance properties can be achieved already at the level of receptive fields and support invariant recognition of objects and events under variations in viewpoint, retinal size, object motion and illumination. The theory can explain the different shapes of receptive field profiles found in biological vision, which are tuned to different sizes and orientations in the image domain as well as to different image velocities in space-time, from a requirement that the visual system should be invariant to the natural types of image transformations that occur in its environment.     

Sensory processing of motor inaccuracy depends on previously performed movement and on subsequent motor corrections: a study of the saccadic system

When goal-directed movements are inaccurate, two responses are generated by the brain: a fast motor correction toward the target and an adaptive motor recalibration developing progressively across subsequent trials. For the saccadic system, there is a clear dissociation between the fast motor correction (corrective saccade production) and the adaptive motor recalibration (primary saccade modification). Error signals used to trigger corrective saccades and to induce adaptation are based on post-saccadic visual feedback. The goal of this study was to determine if similar or different error signals are involved in saccadic adaptation and in corrective saccade generation. Saccadic accuracy was experimentally altered by systematically displacing the visual target during motor execution. Post-saccadic error signals were studied by manipulating visual information in two ways. First, the duration of the displaced target after primary saccade termination was set at 15, 50, 100 or 800 ms in different adaptation sessions. Second, in some sessions, the displaced target was followed by a visual mask that interfered with visual processing. Because they rely on different mechanisms, the adaptation of reactive saccades and the adaptation of voluntary saccades were both evaluated. We found that saccadic adaptation and corrective saccade production were both affected by the manipulations of post-saccadic visual information, but in different ways. This first finding suggests that different types of error signal processing are involved in the induction of these two motor corrections. Interestingly, voluntary saccades required a longer duration of post-saccadic target presentation to reach the same amount of adaptation as reactive saccades. Finally, the visual mask interfered with the production of corrective saccades only during the voluntary saccades adaptation task. These last observations suggest that post-saccadic perception depends on the previously performed action and that the differences between saccade categories of motor correction and adaptation occur at an early level of visual processing.     

Perisaccadic Remapping and Rescaling of Visual Responses in Macaque Superior Colliculus

Visual neurons have spatial receptive fields that encode the positions of objects relative to the fovea. Because foveate animals execute frequent saccadic eye movements, this position information is constantly changing, even though the visual world is generally stationary. Interestingly, visual receptive fields in many brain regions have been found to exhibit changes in strength, size, or position around the time of each saccade, and these changes have often been suggested to be involved in the maintenance of perceptual stability. Crucial to the circuitry underlying perisaccadic changes in visual receptive fields is the superior colliculus (SC), a brainstem structure responsible for integrating visual and oculomotor signals. In this work we have studied the time-course of receptive field changes in the SC. We find that the distribution of the latencies of SC responses to stimuli placed outside the fixation receptive field is bimodal: The first mode is comprised of early responses that are temporally locked to the onset of the visual probe stimulus and stronger for probes placed closer to the classical receptive field. We suggest that such responses are therefore consistent with a perisaccadic rescaling, or enhancement, of weak visual responses within a fixed spatial receptive field. The second mode is more similar to the remapping that has been reported in the cortex, as responses are time-locked to saccade onset and stronger for stimuli placed in the postsaccadic receptive field location. We suggest that these two temporal phases of spatial updating may represent different sources of input to the SC.     

Investigating saccade programming in the praying mantis Tenodera aridifolia using distracter interference paradigms

To investigate the saccadic system in the mantis, I applied distracter interference paradigms. These involved presenting the mantis with a fixation target and one or several distracters supposed to affect saccades towards the target. When a single target was presented, a medium-sized target located in its lower visual field elicited higher rates of saccade response. This preference for target size and position was also observed when a target and a distracter were presented simultaneously. That is, the mantis chose and fixated the target rather than a distracter that was much smaller or larger than the target, or was located above the target. Furthermore, the mantis' preference was not affected by increasing the number of distracters. However, the presence of the distracter decreased the occurrence rate of saccade and increased the response time to saccade. I conclude that distracter interference paradigms are an effective way of investigating the visual processing underlying saccade generation in the mantis. Possible mechanisms of saccade generation in the mantis are discussed.     

Synaptic integration fields and associative plasticity of visual cortical cells in vivo

Two major constraints in connectivity decide the spatial extent of visual cortical receptive fields, both during development and adult functioning: 1) feedforward input, extrinsic to visual cortex, is organized in an orderly projection to form a point-to-point mapping of the retina onto the cortical mantle and constitutes the core of the minimal discharge field (MDF) after amplification by local intracortical circuits; and 2) a second type of connectivity consists of long distance horizontal intracortical connections which are thought to favor the binding of local visual operations occurring simultaneously in different parts of the visual field. We review here possible factors, intrinsic to the considered cortical cell, that may control the effectiveness of post-synaptic integration. Their expression during sensory recognition could depend on the spatio-temporal distribution of active inputs onto the target cell dendrite and on their interplay with non-linearities of the membrane properties. On the basis of intracellular recordings in cat area 17, we have observed that peripheral responses (excitatory and inhibitory) could be boosted by coincident postsynaptic depolarization. This effect is lost if the response and the postsynaptic depolarization are mismatched by 1000 ms, suggesting that temporal correlation of central and peripheral responses could regulate in a non-linear manner the gain of center-surround interactions. This temporal selectivity is compatible with up-regulation of composite potentials by the transient voltage-dependent activation of slowly inactivating conductances in visual cortical neurons. A direct consequence of these different considerations is that the receptive field (RF) of neurons in visual pathways should not be considered as a static hardwired window probing the outer environment, but as an active filter which may continuously adapt and be updated as a function of global context and past experience.     

A reaction-diffusion model to capture disparity selectivity in primary visual cortex

Decades of experimental studies are available on disparity selective cells in visual cortex of macaque and cat. Recently, local disparity map for iso-orientation sites for near-vertical edge preference is reported in area 18 of cat visual cortex. No experiment is yet reported on complete disparity map in V1. Disparity map for layer IV in V1 can provide insight into how disparity selective complex cell receptive field is organized from simple cell subunits. Though substantial amounts of experimental data on disparity selective cells is available, no model on receptive field development of such cells or disparity map development exists in literature. We model disparity selectivity in layer IV of cat V1 using a reaction-diffusion two-eye paradigm. In this model, the wiring between LGN and cortical layer IV is determined by resource an LGN cell has for supporting connections to cortical cells and competition for target space in layer IV. While competing for target space, the same type of LGN cells, irrespective of whether it belongs to left-eye-specific or right-eye-specific LGN layer, cooperate with each other while trying to push off the other type. Our model captures realistic 2D disparity selective simple cell receptive fields, their response properties and disparity map along with orientation and ocular dominance maps. There is lack of correlation between ocular dominance and disparity selectivity at the cell population level. At the map level, disparity selectivity topography is not random but weakly clustered for similar preferred disparities. This is similar to the experimental result reported for macaque. The details of weakly clustered disparity selectivity map in V1 indicate two types of complex cell receptive field organization.     

An intracellular study of space and time representation in primary visual cortical receptive fields

In contrast with previous knowledge based on extracellular recordings, the recent development of intracellular techniques in vivo (sharp electrode or ‘blind patch’) ideally allows experimenters to analyze and dissect the contribution of feedforward and lateral connectivity in the functional expression of a synaptic ‘integration field’. We will present recent data which demonstrate that the visual receptive field of cortical neurons described at the level of subthreshold synaptic events extends over much larger regions of the visual field than previously thought, and that the capacity of cells to amplify subthreshold responses depends on the immediate past history of their membrane potential. Our data suggest that visual cortical receptive fields should not be considered as a fixed entity but more as a dynamic field of integration and association. Two types of dynamics can be argued for: 1) the spatial structure of the minimal discharge field (defined by suprathreshold activation of the cell) can be profoundly reorganized at least during development and most probably during selective phases of learning under the control of activity-dependent mechanisms. Adaptive changes in visual responses are thought to reflect long-lasting potentiation and/or depression of synaptic efficacies conveying ON- and OFF-center information; and 2) during sensory processing, reconfiguration of synaptic weights may be achieved on a much faster time-scale and linke to non-linear properties of the postsynaptic membrane as well as that of recruited networks. Association of information available in the central part of the receptive field (RF) and of input coming from the reputedly ‘unresponsive’ regions surrounding it, or arising simultaneously from different parts of the visual field, might be suppressive in certain cases and capable of boosting hidden responses in other cases, depending on the global stimulus configuration.     

Relationship between level of vigilance and changes in the receptive fields of the cat visual cortex]

The activity of 118 neurones of the primary visual cortex (17th field) was studied in unanesthetized cats, immobilized with d-tubocurarine, in a state of calm wakefulness and in strained attention, alarm. The strained attention was elicited by an air-puff directed to the corner of the closed eye, not used for the photic stimulation. Considerable rearrangments of the receptive field of neurones (94%) were observed in the state of strained attention. In the majority of the studied cortical cells (75%) during 10-30 minutes the excitatory centres of their receptive fields became narrower while their inhibitory periphery widened; in the cells with the inhibitory centres of the receptive fields they were larger in 80% of the cases. It is shown that with dark, mesopic and scotopic adaptation the major characteristics of the changes in the receptive fields remain unaltered. It is assumed that these effects have a behavioural value for the organism, because the created rise in the level of alertness is accompanied by a sharpening of the receptive fields, i.e. by a greater ability to a fine analysis and recognition of visual images.     

End-stopping and the aperture problem: two-dimensional motion signals in macaque V1

Our perception of fine visual detail relies on small receptive fields at early stages of visual processing. However, small receptive fields tend to confound the orientation and velocity of moving edges, leading to ambiguous or inaccurate motion measurements (the aperture problem). Thus, it is often assumed that neurons in primary visual cortex (V1) carry only ambiguous motion information. Here we show that a subpopulation of V1 neurons is capable of signaling motion direction in a manner that is independent of contour orientation. Specifically, end-stopped V1 neurons obtain accurate motion measurements by responding only to the endpoints of long contours, a strategy which renders them largely immune to the aperture problem. Furthermore, the time course of end-stopping is similar to the time course of motion integration by MT neurons. These results suggest that cortical neurons might represent object motion by responding selectively to two-dimensional discontinuities in the visual scene.     

视觉编码的快速学习

活动依赖的神经可塑性在视觉皮层信息处理过程中起着很重要的作用。该文将讲述几个关于视觉刺激引起皮层反应发生快速变化的研究工作。在体膜片钳的实验结果表明,将视觉刺激与能够诱发孽个视皮层神经元发放动作电位的电刺激相偶联可以改变神经元的感受野特性。单电极和多电极胞外记录的实验结果显示,反复地给予自然图形电影刺激,不仅能增加视皮层神经元反应的可靠性,而且能造成之后的自发活动中存在“记忆的痕迹”。最后,用电压敏感染料成像的方法对群体细胞活动进行考察,结果提示视觉活动之后的皮层回放可能是由皮层波介导的。     

How Can Selection of Biologically Inspired Features Improve the Performance of a Robust Object Recognition Model?

Humans can effectively and swiftly recognize objects in complex natural scenes. This outstanding ability has motivated many computational object recognition models. Most of these models try to emulate the behavior of this remarkable system. The human visual system hierarchically recognizes objects in several processing stages. Along these stages a set of features with increasing complexity is extracted by different parts of visual system. Elementary features like bars and edges are processed in earlier levels of visual pathway and as far as one goes upper in this pathway more complex features will be spotted. It is an important interrogation in the field of visual processing to see which features of an object are selected and represented by the visual cortex. To address this issue, we extended a hierarchical model, which is motivated by biology, for different object recognition tasks. In this model, a set of object parts, named patches, extracted in the intermediate stages. These object parts are used for training procedure in the model and have an important role in object recognition. These patches are selected indiscriminately from different positions of an image and this can lead to the extraction of non-discriminating patches which eventually may reduce the performance. In the proposed model we used an evolutionary algorithm approach to select a set of informative patches. Our reported results indicate that these patches are more informative than usual random patches. We demonstrate the strength of the proposed model on a range of object recognition tasks. The proposed model outperforms the original model in diverse object recognition tasks. It can be seen from the experiments that selected features are generally particular parts of target images. Our results suggest that selected features which are parts of target objects provide an efficient set for robust object recognition.     

A computational theory of visual receptive fields

A receptive field constitutes a region in the visual field where a visual cell or a visual operator responds to visual stimuli. This paper presents a theory for what types of receptive field profiles can be regarded as natural for an idealized vision system, given a set of structural requirements on the first stages of visual processing that reflect symmetry properties of the surrounding world. These symmetry properties include (i) covariance properties under scale changes, affine image deformations, and Galilean transformations of space–time as occur for real-world image data as well as specific requirements of (ii) temporal causality implying that the future cannot be accessed and (iii) a time-recursive updating mechanism of a limited temporal buffer of the past as is necessary for a genuine real-time system. Fundamental structural requirements are also imposed to ensure (iv) mutual consistency and a proper handling of internal representations at different spatial and temporal scales. It is shown how a set of families of idealized receptive field profiles can be derived by necessity regarding spatial, spatio-chromatic, and spatio-temporal receptive fields in terms of Gaussian kernels, Gaussian derivatives, or closely related operators. Such image filters have been successfully used as a basis for expressing a large number of visual operations in computer vision, regarding feature detection, feature classification, motion estimation, object recognition, spatio-temporal recognition, and shape estimation. Hence, the associated so-called scale-space theory constitutes a both theoretically well-founded and general framework for expressing visual operations. There are very close similarities between receptive field profiles predicted from this scale-space theory and receptive field profiles found by cell recordings in biological vision. Among the family of receptive field profiles derived by necessity from the assumptions, idealized models with very good qualitative agreement are obtained for (i) spatial on-center/off-surround and off-center/on-surround receptive fields in the fovea and the LGN, (ii) simple cells with spatial directional preference in V1, (iii) spatio-chromatic double-opponent neurons in V1, (iv) space–time separable spatio-temporal receptive fields in the LGN and V1, and (v) non-separable space–time tilted receptive fields in V1, all within the same unified theory. In addition, the paper presents a more general framework for relating and interpreting these receptive fields conceptually and possibly predicting new receptive field profiles as well as for pre-wiring covariance under scaling, affine, and Galilean transformations into the representations of visual stimuli. This paper describes the basic structure of the necessity results concerning receptive field profiles regarding the mathematical foundation of the theory and outlines how the proposed theory could be used in further studies and modelling of biological vision. It is also shown how receptive field responses can be interpreted physically, as the superposition of relative variations of surface structure and illumination variations, given a logarithmic brightness scale, and how receptive field measurements will be invariant under multiplicative illumination variations and exposure control mechanisms.     

Right-hemispheric dominance for visual remapping in humans

We review evidence showing a right-hemispheric dominance for visuo-spatial processing and representation in humans. Accordingly, visual disorganization symptoms (intuitively related to remapping impairments) are observed in both neglect and constructional apraxia. More specifically, we review findings from the intervening saccade paradigm in humans--and present additional original data--which suggest a specific role of the asymmetrical network at the temporo-parietal junction (TPJ) in the right hemisphere in visual remapping: following damage to the right dorsal posterior parietal cortex (PPC) as well as part of the corpus callosum connecting the PPC to the frontal lobes, patient OK in a double-step saccadic task exhibited an impairment when the second saccade had to be directed rightward. This singular and lateralized deficit cannot result solely from the patient's cortical lesion and, therefore, we propose that it is due to his callosal lesion that may specifically interrupt the interhemispheric transfer of information necessary to execute accurate rightward saccades towards a remapped target location. This suggests a specialized right-hemispheric network for visuo-spatial remapping that subsequently transfers target location information to downstream planning regions, which are symmetrically organized.     

Learning visuomotor transformations for gaze-control and grasping

For reaching to and grasping of an object, visual information about the object must be transformed into motor or postural commands for the arm and hand. In this paper, we present a robot model for visually guided reaching and grasping. The model mimics two alternative processing pathways for grasping, which are also likely to coexist in the human brain. The first pathway directly uses the retinal activation to encode the target position. In the second pathway, a saccade controller makes the eyes (cameras) focus on the target, and the gaze direction is used instead as positional input. For both pathways, an arm controller transforms information on the target’s position and orientation into an arm posture suitable for grasping. For the training of the saccade controller, we suggest a novel staged learning method which does not require a teacher that provides the necessary motor commands. The arm controller uses unsupervised learning: it is based on a density model of the sensor and the motor data. Using this density, a mapping is achieved by completing a partially given sensorimotor pattern. The controller can cope with the ambiguity in having a set of redundant arm postures for a given target. The combined model of saccade and arm controller was able to fixate and grasp an elongated object with arbitrary orientation and at arbitrary position on a table in 94% of trials.     

Characteristics of dynamic reorganization of receptive fields of cortical and geniculate visual neurons of cat in response to changes in photic stimulation parameters

The effect of parameters of local photic stimulation of different points of the receptive field on the characteristics of dynamic reorganization of receptive fields of cortical and geniculate visual neurons within microintervals of time observed previously was studied in computer-controlled experiments on unanesthetized, curarized cats. Dependence on the degree of widening of the receptive field and the temporal characteristics of this process on the background illumination level, intensity, contrast, area, duration, energy, and orientation of a local rectangular or circular photic stimulus flashing in random order at 100 points of the tested part of the visual field was studied. It was concluded that the background illumination level and the intensity, size, duration, and orientation of the stimulus have a specific effect on dynamic reorganization of the receptive field. The effects of all the parameters studied on the dynamics of the receptive field were shown to be nonlinear functions with optimal values that differed for different cells. The possible functional role of this effect and also the probability that it may participate in information coding in the visual system are discussed.Institute of Higher Nervous Activity and Neurophysiology, Academy of Sciences of the USSR, Moscow. Translated from Neirofiziologiya, Vol. 15, No. 4, pp. 339–346, July–August, 1983.