Vertical Binocular Disparity is Encoded Implicitly within a Model Neuronal Population Tuned to Horizontal Disparity and Orientation

Primary visual cortex is often viewed as a “cyclopean retina”, performing the initial encoding of binocular disparities between left and right images. Because the eyes are set apart horizontally in the head, binocular disparities are predominantly horizontal. Yet, especially in the visual periphery, a range of non-zero vertical disparities do occur and can influence perception. It has therefore been assumed that primary visual cortex must contain neurons tuned to a range of vertical disparities. Here, I show that this is not necessarily the case. Many disparity-selective neurons are most sensitive to changes in disparity orthogonal to their preferred orientation. That is, the disparity tuning surfaces, mapping their response to different two-dimensional (2D) disparities, are elongated along the cell''s preferred orientation. Because of this, even if a neuron''s optimal 2D disparity has zero vertical component, the neuron will still respond best to a non-zero vertical disparity when probed with a sub-optimal horizontal disparity. This property can be used to decode 2D disparity, even allowing for realistic levels of neuronal noise. Even if all V1 neurons at a particular retinotopic location are tuned to the expected vertical disparity there (for example, zero at the fovea), the brain could still decode the magnitude and sign of departures from that expected value. This provides an intriguing counter-example to the common wisdom that, in order for a neuronal population to encode a quantity, its members must be tuned to a range of values of that quantity. It demonstrates that populations of disparity-selective neurons encode much richer information than previously appreciated. It suggests a possible strategy for the brain to extract rarely-occurring stimulus values, while concentrating neuronal resources on the most commonly-occurring situations.     

Stereoscopic Depth Perception Using a Model Based on the Primary Visual Cortex

This work describes an approach inspired by the primary visual cortex using the stimulus response of the receptive field profiles of binocular cells for disparity computation. Using the energy model based on the mechanism of log-Gabor filters for disparity encodings, we propose a suitable model to consistently represent the complex cells by computing the wide bandwidths of the cortical cells. This way, the model ensures the general neurophysiological findings in the visual cortex (V1), emphasizing the physical disparities and providing a simple selection method for the complex cell response. The results suggest that our proposed approach can achieve better results than a hybrid model with phase-shift and position-shift using position disparity alone.     

视差检测：简单细胞、复杂细胞及能量模型

立体视觉信息的处理在于皮层双眼性细胞的活动.皮层中简单细胞对视差的编码方式被认为有两种：位置差（position shift）和相位差（phase shift）,但简单细胞并不适合作为视差检测器.对一些复杂细胞的视差响应特性的生理研究,发现复杂细胞是一种比较适合的视差检测器.模型的研究提出基于这类简单细胞的复杂细胞能量模型,可以很好的检测视差,并可以较好的解释一些生理现象.     

The development of the binocular depth cells in the secondary visual cortex of the lamb.

In most respects, the response properties of cells in the secondary visual cortex of the newborn lamb were indistinguishable from those in the adult. The cells were sharply selective to orientation; the orientation preferences were the same in each eye, and they varied systematically as the electrode penetrated the cortex. The receptive-field organization did not differ noticeably from that in adults, and complex, hypercomplex, and a few simple cells were all observed. The ocular dominance distribution was similar to that in the adult. Most importantly, binocular cells were found with disparate receptive fields even in newborn, visually inexperienced animals. As in the adult, the disparities were largely horizontal, and they appeared to be arranged in columns. Many of the cells responded preferentially to a binocular stimulus at a particular disparity setting (often approximately zero), but unlike those in the adult almost all the binocular cells in the newborn lamb would also respond monocularly, and the enhancement at the optimal disparity was less than in the adult. The full development of binocular selectivity took several weeks, and was blocked by binocular deprivation. We conclude that the basic wiring of stereoscopic mechanisms is innate, but the development of mature binocular interaction may depend on an adaptive process which makes use of the visual information received during binocular stimulation.     

Stereoscopic Vision in the Absence of the Lateral Occipital Cortex

Both dorsal and ventral cortical visual streams contain neurons sensitive to binocular disparities, but the two streams may underlie different aspects of stereoscopic vision. Here we investigate stereopsis in the neurological patient D.F., whose ventral stream, specifically lateral occipital cortex, has been damaged bilaterally, causing profound visual form agnosia. Despite her severe damage to cortical visual areas, we report that DF''s stereo vision is strikingly unimpaired. She is better than many control observers at using binocular disparity to judge whether an isolated object appears near or far, and to resolve ambiguous structure-from-motion. DF is, however, poor at using relative disparity between features at different locations across the visual field. This may stem from a difficulty in identifying the surface boundaries where relative disparity is available. We suggest that the ventral processing stream may play a critical role in enabling healthy observers to extract fine depth information from relative disparities within one surface or between surfaces located in different parts of the visual field.     

A reaction-diffusion model to capture disparity selectivity in primary visual cortex

Decades of experimental studies are available on disparity selective cells in visual cortex of macaque and cat. Recently, local disparity map for iso-orientation sites for near-vertical edge preference is reported in area 18 of cat visual cortex. No experiment is yet reported on complete disparity map in V1. Disparity map for layer IV in V1 can provide insight into how disparity selective complex cell receptive field is organized from simple cell subunits. Though substantial amounts of experimental data on disparity selective cells is available, no model on receptive field development of such cells or disparity map development exists in literature. We model disparity selectivity in layer IV of cat V1 using a reaction-diffusion two-eye paradigm. In this model, the wiring between LGN and cortical layer IV is determined by resource an LGN cell has for supporting connections to cortical cells and competition for target space in layer IV. While competing for target space, the same type of LGN cells, irrespective of whether it belongs to left-eye-specific or right-eye-specific LGN layer, cooperate with each other while trying to push off the other type. Our model captures realistic 2D disparity selective simple cell receptive fields, their response properties and disparity map along with orientation and ocular dominance maps. There is lack of correlation between ocular dominance and disparity selectivity at the cell population level. At the map level, disparity selectivity topography is not random but weakly clustered for similar preferred disparities. This is similar to the experimental result reported for macaque. The details of weakly clustered disparity selectivity map in V1 indicate two types of complex cell receptive field organization.     

Receptive fields of disparity-tuned simple cells in macaque V1

Binocular simple cells in primary visual cortex (V1) are the first cells along the mammalian visual pathway to receive input from both eyes. Two models of how binocular simple cells could extract disparity information have been put forward. The phase-shift model proposes that the receptive fields in the two eyes have different subunit organizations, while the position-shift model proposes that they have different overall locations. In five fixating macaque monkeys, we recorded from 30 disparity-tuned simple cells that showed selectivity to the disparity in a random dot stereogram. High-resolution maps of the left and right eye receptive fields indicated that both phase and position shifts were common. Single cells usually showed a combination of the two, and the optimum disparity was best correlated with the sum of receptive field phase and position shift.     

Visuomotor transformation in the fly gaze stabilization system

For sensory signals to control an animal's behavior, they must first be transformed into a format appropriate for use by its motor systems. This fundamental problem is faced by all animals, including humans. Beyond simple reflexes, little is known about how such sensorimotor transformations take place. Here we describe how the outputs of a well-characterized population of fly visual interneurons, lobula plate tangential cells (LPTCs), are used by the animal's gaze-stabilizing neck motor system. The LPTCs respond to visual input arising from both self-rotations and translations of the fly. The neck motor system however is involved in gaze stabilization and thus mainly controls compensatory head rotations. We investigated how the neck motor system is able to selectively extract rotation information from the mixed responses of the LPTCs. We recorded extracellularly from fly neck motor neurons (NMNs) and mapped the directional preferences across their extended visual receptive fields. Our results suggest that-like the tangential cells-NMNs are tuned to panoramic retinal image shifts, or optic flow fields, which occur when the fly rotates about particular body axes. In many cases, tangential cells and motor neurons appear to be tuned to similar axes of rotation, resulting in a correlation between the coordinate systems the two neural populations employ. However, in contrast to the primarily monocular receptive fields of the tangential cells, most NMNs are sensitive to visual motion presented to either eye. This results in the NMNs being more selective for rotation than the LPTCs. Thus, the neck motor system increases its rotation selectivity by a comparatively simple mechanism: the integration of binocular visual motion information.     

Parallel processing of binocular disparity in the cat's retinogeniculocortical pathways

In the cat, parallel streams of information processing have been traced from X-, Y- and W-type retinal ganglion cells to visual cortical areas 17 (X-, Y- and W-type), 18 (Y-type) and 19 (W-type). In the present study we have examined, in the anaesthetized and paralysed adult cat, the role played by X-, Y- and W-subsystems, projecting to areas 17 and 19, in the processing of binocular retinal disparity. The tapetal reflection technique was used to monitor residual eye movements and to provide a map, for each eye, of the retinal blood vessels which could later be compared with retinal wholemounts stained with cresyl violet to reveal the area centralis. The receptive-field disparities of cells recorded from areas 17 and 19 were compared with each other and with reference to the visual axes defined by the area centralis of each eye. Cells of area 19 (receiving W-type input) had horizontal receptive-field disparities that were significantly more divergent than those of the cells in area 17 and 17-18 'border region'. Referred to the area centralis, the mean horizontal receptive-field disparity in area 19 was -0.5 degrees (+/- 0.8 degrees). The mean horizontal receptive-field disparity of area 17 (receiving X-, Y- and W-type input) was convergent with respect to the visual axis at +2 degrees (+/- 0.5 degrees). Finally, the mean horizontal receptive-field disparity of the cells in the 17-18 border region (which receive mainly Y-type input) was even more convergent (2.6 degrees +/- 1.5 degrees) than that of area 17. Binocular interactions of cortical neurons were tested with the Risley biprism technique. Area 19 cells had maximal responses to binocular stimulation when the receptive-field disparities were either close to zero or slightly divergent. In contrast, area 17 cells tended to respond optimally to disparities that were either slightly or strongly convergent. At the level of the lateral geniculate nucleus there were significant differences between the receptive-field disparities inferred from the comparison of receptive-field positions of adjacent neurons recorded on either side of the border between the A and A1 geniculate laminae and those inferred from a similar comparison at the C1-C2 border. The mean horizontal disparities inferred from the interlaminar comparison at the A-A1 border were +2.1 degrees (+/- 0.3 degrees); those inferred from the interlaminar comparison at the C1-C2 border -0.2 (+/- 0.2 degrees) were more divergent.(ABSTRACT TRUNCATED AT 400 WORDS)     

Spatial stereoresolution for depth corrugations may be set in primary visual cortex

Stereo "3D" depth perception requires the visual system to extract binocular disparities between the two eyes' images. Several current models of this process, based on the known physiology of primary visual cortex (V1), do this by computing a piecewise-frontoparallel local cross-correlation between the left and right eye's images. The size of the "window" within which detectors examine the local cross-correlation corresponds to the receptive field size of V1 neurons. This basic model has successfully captured many aspects of human depth perception. In particular, it accounts for the low human stereoresolution for sinusoidal depth corrugations, suggesting that the limit on stereoresolution may be set in primary visual cortex. An important feature of the model, reflecting a key property of V1 neurons, is that the initial disparity encoding is performed by detectors tuned to locally uniform patches of disparity. Such detectors respond better to square-wave depth corrugations, since these are locally flat, than to sinusoidal corrugations which are slanted almost everywhere. Consequently, for any given window size, current models predict better performance for square-wave disparity corrugations than for sine-wave corrugations at high amplitudes. We have recently shown that this prediction is not borne out: humans perform no better with square-wave than with sine-wave corrugations, even at high amplitudes. The failure of this prediction raised the question of whether stereoresolution may actually be set at later stages of cortical processing, perhaps involving neurons tuned to disparity slant or curvature. Here we extend the local cross-correlation model to include existing physiological and psychophysical evidence indicating that larger disparities are detected by neurons with larger receptive fields (a size/disparity correlation). We show that this simple modification succeeds in reconciling the model with human results, confirming that stereoresolution for disparity gratings may indeed be limited by the size of receptive fields in primary visual cortex.     

Vertical disparity, egocentric distance and stereoscopic depth constancy: a new interpretation

There has long been a problem concerning the presence in the visual cortex of binocularly activated cells that are selective for vertical stimulus disparities because it is generally believed that only horizontal disparities contribute to stereoscopic depth perception. The accepted view is that stereoscopic depth estimates are only relative to the fixation point and that independent information from an extraretinal source is needed to scale for absolute or egocentric distance. Recently, however, theoretical computations have shown that egocentric distance can be estimated directly from vertical disparities without recourse to extraretinal sources. There has been little impetus to follow up these computations with experimental observations, because the vertical disparities that normally occur between the images in the two eyes have always been regarded as being too small to be of significance for visual perception and because experiments have consistently shown that our conscious appreciation of egocentric distance is rather crude and unreliable. Nevertheless, the veridicality of stereoscopic depth constancy indicates that accurate distance information is available to the visual system and that the information about egocentric distance and horizontal disparity are processed together so as to continually recalibrate the horizontal disparity values for different absolute distances. Computations show that the recalibration can be based directly on vertical disparities without the need for any intervening estimates of absolute distance. This may partly explain the relative crudity of our conscious appreciation of egocentric distance. From published data it has been possible to calculate the magnitude of the vertical disparities that the human visual system must be able to discriminate in order for depth constancy to have the observed level of veridicality. From published data on the induced effect it has also been possible to calculate the threshold values for the detection of vertical disparities by the visual system. These threshold values are smaller than those needed to provide for the recalibration of the horizontal disparities in the interests of veridical depth constancy. An outline is given of the known properties of the binocularly activated cells in the striate cortex that are able to discriminate and assess the vertical disparities. Experiments are proposed that should validate, or otherwise, the concepts put forward in this paper.     

Size-disparity correlation in human binocular depth perception.

To use the small horizontal disparities between images projected to the eyes for the recovery of three-dimensional information, our visual system must first identify which feature in one eye's image corresponds with which in the other. The earliest level of disparity processing in primates (V1) contains cells that are spatial-frequency tuned. If such cells have a disparity range that covers only a single period of their mean tuning frequency, there will always be exactly one potential match within this range. Here, this 'size-disparity' hypothesis was tested by measuring the contrast sensitivity of stereopsis as a function of disparity for single bandpass-filtered items. It was found that thresholds were low and relatively constant up to disparities an order of magnitude larger than is predicted by this constraint. Furthermore, peak sensitivity was relatively independent of spatial frequency. A control experiment showed that binocular correlation of the carrier is necessary for this task. In a third experiment, the maximum disparity that supports threshold performance was compared for an isolated bandpass item and bandpass-filtered noise. This limit was found to be five times larger for the isolated stimuli. In summary, these findings show that the initial stage of disparity detection is not limited by the size-disparity constraint. For stimuli with multiple false targets, however, processes subsequent to this stage reduce the disparity range over which the correspondence problem can be solved.     

A map for horizontal disparity in monkey V2

The perception of visual depth is determined by integration of spatial disparities of inputs from the two eyes. Single cells in visual cortex of monkeys are known to respond to specific binocular disparities; however, little is known about their functional organization. We now show, using intrinsic signal optical imaging and single-unit physiology, that, in the thick stripe compartments of the second visual area (V2), there is a clustered organization of Near cells and Far cells, and moreover, there are topographic maps for Near to Far disparities within V2. Our findings suggest that maps for visual disparity are calculated in V2, and demonstrate parallels in functional organization between the thin, pale, and thick stripes of V2.     

The Role of Binocular Disparity in Stereoscopic Images of Objects in the Macaque Anterior Intraparietal Area

Neurons in the macaque Anterior Intraparietal area (AIP) encode depth structure in random-dot stimuli defined by gradients of binocular disparity, but the importance of binocular disparity in real-world objects for AIP neurons is unknown. We investigated the effect of binocular disparity on the responses of AIP neurons to images of real-world objects during passive fixation. We presented stereoscopic images of natural and man-made objects in which the disparity information was congruent or incongruent with disparity gradients present in the real-world objects, and images of the same objects where such gradients were absent. Although more than half of the AIP neurons were significantly affected by binocular disparity, the great majority of AIP neurons remained image selective even in the absence of binocular disparity. AIP neurons tended to prefer stimuli in which the depth information derived from binocular disparity was congruent with the depth information signaled by monocular depth cues, indicating that these monocular depth cues have an influence upon AIP neurons. Finally, in contrast to neurons in the inferior temporal cortex, AIP neurons do not represent images of objects in terms of categories such as animate-inanimate, but utilize representations based upon simple shape features including aspect ratio.     

Disparity estimation through Green’s functions of matching equations

Binocular disparities arise from positional differences of scene features projected in the two retinae, and constitute the primary sensory cue for stereo vision. Here we introduce a new computational model for disparity estimation, based on the Green’s function of an image matching equation. When filtering a Gabor-function-modulated signal, the considered Green’s function yields a similarly modulated but shifted version of the original signal. Since a Gabor function models the receptive field of a cortical simple cell, the Green’s kernel thus allows the simulation of relative shifts between the cell’s left and right binocular inputs. A measure of the local degree of matching of such shifted inputs can then be introduced which affords disparity estimation in a similar manner to the energy model of the complex cortical cells. We have therefore effectively reformulated, in physiologically plausible terms, an image matching approach to disparity estimation. Our experiments show that the Green’s function method allows the detection of disparities both from random-dot and real-world stereograms. Partially supported by CNPq-Brazil.     

Stereoscopic correspondence for ambiguous targets is affected by elevation and fixation distance

Binocular correspondence must be determined if disparity is to be used to provide information about three-dimensional shape. The current study investigated whether knowledge of the statistical distribution of disparities in the natural environment is employed in this process. A simple model, which produces distributions of distances similar to those found in the natural environment, was used to predict the distribution of disparities in natural images. This model predicts that crossed disparities will be more likely as (i) stimulus elevation decreases below fixation and (ii) fixation distance increases. To determine whether these factors influence binocular correspondence for human observers, ambiguous stereograms were presented to observers, as stimulus elevation and fixation distance were manipulated. Clear biases were observed in the depth perceived in these stereograms, which were more likely to be seen as closer than fixation (i) for stimuli presented below fixation and (ii) as fixation distance increased. These results suggest that binocular correspondence is determined in a manner consistent with the distributions of disparities expected in natural scenes.     

频差在立体视觉信息加工中的作用 Ⅱ、频差“克服”视差

双眼立体视觉机制至今不很清楚,存在不少争论,研究它具有深远意义。我们的兴趣是从心理物理、电生理和理论模型三方面开展工作,最终目标是试图搞清楚视觉立体信息处理类机制。本文主要利用心理物理学方法研究频差克差视差的问题。我们利用自己研制的一种多功能立体图形发生器产生左边为非均匀条纹、右边为均匀条纹的一系列具有不同视差的立体图对。在感知到“阶梯”后,用三种方法使得“阶梯”感变平:①改变均匀条纹的频率,②改变均匀条纹与被试的距离,③改变非均匀条纹与被试者的距离。从而实现了频差“克服”视差。我们的结果支持用频差来解释双眼倾斜现象,它使我们相信频差是视差在初级视系统中的表象形式。     

频差在立体视觉信息加工中的作用 Ⅱ、频差“克服”视差

双眼立体视觉机制至今不很清楚,存在不少争论,研究它具有深远意义。我们的兴趣是从心理物理、电生理和理论模型三方面开展工作,最终目标是试图搞清楚视觉立体信息处理类机制。本文主要利用心理物理学方法研究频差克差视差的问题。我们利用自己研制的一种多功能立体图形发生器产生左边为非均匀条纹、右边为均匀条纹的一系列具有不同视差的立体图对。在感知到“阶梯”后,用三种方法使得“阶梯”感变平:①改变均匀条纹的频率,②改变均匀条纹与被试的距离,③改变非均匀条纹与被试者的距离。从而实现了频差“克服”视差。我们的结果支持用频差来解释双眼倾斜现象,它使我们相信频差是视差在初级视系统中的表象形式。     

Brain activation difference evoked by different binocular disparities of stereograms: An fMRI study

The binocular disparity of two retina images is a main cue of stereoscopic vision. However, the global dependency between brain response and binocular disparity still remains unclear. Here, we used functional Magnetic Resonance Imaging (fMRI) to identify stereopsis-related brain regions with a modified Random Dot Stereogram (RDS) and plotted the activation variation curves under different disparity size. In order to eliminate the confounding shape difference between the stereogram and the plane, commonly seen in RDS, we modified the RDS to a checkerboard version. We found that V3A, V7 and MT+/V5 in dorsal visual stream were activated in stereoscopic experiment, while little activation was found in ventral visual regions. According to the activation trends, 13 subjects were divided into three groups: 5 subjects with turning points (a shift from increased to decreased activation), 5 subjects without turning points and 3 subjects with activation unrelated to disparity. We inferred that the dorsal visual stream primarily processes spatial depth information, rather than shape information.