Blur and disparity are complementary cues to depth

Estimating depth from binocular disparity is extremely precise, and the cue does not depend on statistical regularities in the environment. Thus, disparity is commonly regarded as the best visual cue for determining 3D layout. But depth from disparity is only precise near where one is looking; it is quite imprecise elsewhere. Away from fixation, vision resorts to using other depth cues-e.g., linear perspective, familiar size, aerial perspective. But those cues depend on statistical regularities in the environment and are therefore not always reliable. Depth from defocus blur relies on fewer assumptions and has the same geometric constraints as disparity but different physiological constraints. Blur could in principle fill in the parts of visual space where disparity is imprecise. We tested this possibility with a depth-discrimination experiment. Disparity was more precise near fixation and blur was indeed more precise away from fixation. When both cues were available, observers relied on the more informative one. Blur appears to play an important, previously unrecognized role in depth perception. Our findings lead to a new hypothesis about the evolution of slit-shaped pupils and have implications for the design and implementation of stereo 3D displays.     

Does vertical disparity scale the perception of stereoscopic depth?

It has been suggested that a measure of the gradients of vertical disparity over a surface may scale the mapping between horizontal disparity and perceived depth. We have investigated this possibility by obtaining estimates of the depth within stereograms that simulated two apposed fronto-parallel planes placed at different distances from an observer. The gradients of vertical disparity in a stereogram were set to simulate those appropriate to a viewing distance of 12.5 cm, 25 cm, 50 cm or 100 cm, whereas the distance specified by vergence and accommodative cues was always fixed at 50 cm. Judgements of the perceived depth between the two planes were uninfluenced by changes in the gradients of vertical disparity. It thus seems that the human visual system does not employ vertical disparity as a scaling parameter in stereoscopic depth judgements.     

Effect of pictorial depth cues, binocular disparity cues and motion parallax depth cues on lightness perception in three-dimensional virtual scenes

Background

Surface lightness perception is affected by scene interpretation. There is some experimental evidence that perceived lightness under bi-ocular viewing conditions is different from perceived lightness in actual scenes but there are also reports that viewing conditions have little or no effect on perceived color. We investigated how mixes of depth cues affect perception of lightness in three-dimensional rendered scenes containing strong gradients of illumination in depth.

Methodology/Principal Findings

Observers viewed a virtual room (4 m width×5 m height×17.5 m depth) with checkerboard walls and floor. In four conditions, the room was presented with or without binocular disparity (BD) depth cues and with or without motion parallax (MP) depth cues. In all conditions, observers were asked to adjust the luminance of a comparison surface to match the lightness of test surfaces placed at seven different depths (8.5–17.5 m) in the scene. We estimated lightness versus depth profiles in all four depth cue conditions. Even when observers had only pictorial depth cues (no MP, no BD), they partially but significantly discounted the illumination gradient in judging lightness. Adding either MP or BD led to significantly greater discounting and both cues together produced the greatest discounting. The effects of MP and BD were approximately additive. BD had greater influence at near distances than far.

Conclusions/Significance

These results suggest the surface lightness perception is modulated by three-dimensional perception/interpretation using pictorial, binocular-disparity, and motion-parallax cues additively. We propose a two-stage (2D and 3D) processing model for lightness perception.     

The Role of Binocular Disparity in Stereoscopic Images of Objects in the Macaque Anterior Intraparietal Area

Neurons in the macaque Anterior Intraparietal area (AIP) encode depth structure in random-dot stimuli defined by gradients of binocular disparity, but the importance of binocular disparity in real-world objects for AIP neurons is unknown. We investigated the effect of binocular disparity on the responses of AIP neurons to images of real-world objects during passive fixation. We presented stereoscopic images of natural and man-made objects in which the disparity information was congruent or incongruent with disparity gradients present in the real-world objects, and images of the same objects where such gradients were absent. Although more than half of the AIP neurons were significantly affected by binocular disparity, the great majority of AIP neurons remained image selective even in the absence of binocular disparity. AIP neurons tended to prefer stimuli in which the depth information derived from binocular disparity was congruent with the depth information signaled by monocular depth cues, indicating that these monocular depth cues have an influence upon AIP neurons. Finally, in contrast to neurons in the inferior temporal cortex, AIP neurons do not represent images of objects in terms of categories such as animate-inanimate, but utilize representations based upon simple shape features including aspect ratio.     

频差在立体视觉信息加工中的作用 Ⅱ、频差“克服”视差

双眼立体视觉机制至今不很清楚,存在不少争论,研究它具有深远意义。我们的兴趣是从心理物理、电生理和理论模型三方面开展工作,最终目标是试图搞清楚视觉立体信息处理类机制。本文主要利用心理物理学方法研究频差克差视差的问题。我们利用自己研制的一种多功能立体图形发生器产生左边为非均匀条纹、右边为均匀条纹的一系列具有不同视差的立体图对。在感知到“阶梯”后,用三种方法使得“阶梯”感变平:①改变均匀条纹的频率,②改变均匀条纹与被试的距离,③改变非均匀条纹与被试者的距离。从而实现了频差“克服”视差。我们的结果支持用频差来解释双眼倾斜现象,它使我们相信频差是视差在初级视系统中的表象形式。     

频差在立体视觉信息加工中的作用 Ⅱ、频差“克服”视差

双眼立体视觉机制至今不很清楚,存在不少争论,研究它具有深远意义。我们的兴趣是从心理物理、电生理和理论模型三方面开展工作,最终目标是试图搞清楚视觉立体信息处理类机制。本文主要利用心理物理学方法研究频差克差视差的问题。我们利用自己研制的一种多功能立体图形发生器产生左边为非均匀条纹、右边为均匀条纹的一系列具有不同视差的立体图对。在感知到“阶梯”后,用三种方法使得“阶梯”感变平:①改变均匀条纹的频率,②改变均匀条纹与被试的距离,③改变非均匀条纹与被试者的距离。从而实现了频差“克服”视差。我们的结果支持用频差来解释双眼倾斜现象,它使我们相信频差是视差在初级视系统中的表象形式。     

Shading Beats Binocular Disparity in Depth from Luminance Gradients: Evidence against a Maximum Likelihood Principle for Cue Combination

Perceived depth is conveyed by multiple cues, including binocular disparity and luminance shading. Depth perception from luminance shading information depends on the perceptual assumption for the incident light, which has been shown to default to a diffuse illumination assumption. We focus on the case of sinusoidally corrugated surfaces to ask how shading and disparity cues combine defined by the joint luminance gradients and intrinsic disparity modulation that would occur in viewing the physical corrugation of a uniform surface under diffuse illumination. Such surfaces were simulated with a sinusoidal luminance modulation (0.26 or 1.8 cy/deg, contrast 20%-80%) modulated either in-phase or in opposite phase with a sinusoidal disparity of the same corrugation frequency, with disparity amplitudes ranging from 0’-20’. The observers’ task was to adjust the binocular disparity of a comparison random-dot stereogram surface to match the perceived depth of the joint luminance/disparity-modulated corrugation target. Regardless of target spatial frequency, the perceived target depth increased with the luminance contrast and depended on luminance phase but was largely unaffected by the luminance disparity modulation. These results validate the idea that human observers can use the diffuse illumination assumption to perceive depth from luminance gradients alone without making an assumption of light direction. For depth judgments with combined cues, the observers gave much greater weighting to the luminance shading than to the disparity modulation of the targets. The results were not well-fit by a Bayesian cue-combination model weighted in proportion to the variance of the measurements for each cue in isolation. Instead, they suggest that the visual system uses disjunctive mechanisms to process these two types of information rather than combining them according to their likelihood ratios.     

Brain activation difference evoked by different binocular disparities of stereograms: An fMRI study

The binocular disparity of two retina images is a main cue of stereoscopic vision. However, the global dependency between brain response and binocular disparity still remains unclear. Here, we used functional Magnetic Resonance Imaging (fMRI) to identify stereopsis-related brain regions with a modified Random Dot Stereogram (RDS) and plotted the activation variation curves under different disparity size. In order to eliminate the confounding shape difference between the stereogram and the plane, commonly seen in RDS, we modified the RDS to a checkerboard version. We found that V3A, V7 and MT+/V5 in dorsal visual stream were activated in stereoscopic experiment, while little activation was found in ventral visual regions. According to the activation trends, 13 subjects were divided into three groups: 5 subjects with turning points (a shift from increased to decreased activation), 5 subjects without turning points and 3 subjects with activation unrelated to disparity. We inferred that the dorsal visual stream primarily processes spatial depth information, rather than shape information.     

Performance data indicate summation for pictorial depth-cues in slanted surfaces

Over recent years much has been learned about the way in which depth cues are combined (e.g. Landy et al.. 1995). The majority of this work has used subjective measures, a rating scale or a point of subjective equality, to deduce the relative contributions of different cues to perception. We have adopted a very different approach by using two interval forced-choice (21FC) performance measures and a signal processing framework. We performed summation experiments for depth cue increment thresholds between pairs of pictorial depth cues in displays depicting slanted planar surfaces made from arrays of circular 'contrast' elements. Summation was found to be ideal when size-gradient was paired with contrast-gradient for a wide range of depth-gradient magnitudes in the null stimulus. For a pairing of size-gradient and linear perspective, substantial summation (> 1.5 dB) was found only when the null stimulus had intermediate depth gradients; when flat or steeply inclined surfaces were depicted, summation was diminished or abolished. Summation was also abolished when one of the target cues was (i) not a depth cue, or (ii) added in conflict. We conclude that vision has a depth mechanism for the constructive combination of pictorial depth cues and suggest two generic models of summation to describe the results. Using similar psychophysical methods. Bradshaw and Rogers (1996) revealed a mechanism for the depth cues of motion parallax and binocular disparity. Whether this is the same or a different mechanism from the one reported here awaits elaboration.     

Interrelation of kinetic and stereoscopic depth: behavior and physiology in vertebrates

The target article gathers compelling behavioral evidence that motion parallax provides depth information in a variety of animal species. A more general evaluation of kinetic depth cues subserving depth perception would call attention to recent studies in monkeys, demonstrating the interrelation of kinetic and stereoscopic depth cues both on a behavioral and physiological level. Furthermore, it is argued that binocularity in birds has a clear function in stereopsis.     

Extrapolating and interpolating surfaces in depth

Random-dot stereograms were generated with a blank area placed in part of the right-hand image so making a patchwork of monocular and binocular areas. The perceived depth and shape of the monocular region, where depth was not explicitly marked, depended in part on the depth and surface orientation of adjacent binocular areas. Thus a monocular rectangle flanked by two binocular rectangles which were placed in different fronto-parallel planes was seen as a sloping surface spanning the depth between the binocular regions, and, under some conditions, the gradient of a sloping binocular plane extended into a neighbouring monocular area. Division of the monocular region into two by textural discontinuities or discontinuities of motion sometimes altered the shape of the extrapolated surface. Often, though, the shape was unchanged by such discontinuities implying that both two- and three-dimensional features are used to segment a scene into separate surfaces. Pictorial cues also contribute to the shape and apparent depth of the monocular surface. For instance, when subjects viewed a display consisting of portions of a cube of which two ends were shown stereoscopically and one side monocularly, the monocular side was seen in three dimensions filling the gap between the ends. When stereo cues were pitted against pictorial cues, sometimes pictorial cues and sometimes stereo cues dominated, and sometimes the surface contained sharp discontinuities enabling both to be accommodated.     

Stereoscopic vision: what's the first step?

Neurons in primary visual cortex respond to binocular disparity, the raw material of stereoscopic depth perception. Although these neurons are probably essential to depth perception, a recent study has shown that they are unable to compute depth itself.     

Binocular coordination: reading stereoscopic sentences in depth

The present study employs a stereoscopic manipulation to present sentences in three dimensions to subjects as they read for comprehension. Subjects read sentences with (a) no depth cues, (b) a monocular depth cue that implied the sentence loomed out of the screen (i.e., increasing retinal size), (c) congruent monocular and binocular (retinal disparity) depth cues (i.e., both implied the sentence loomed out of the screen) and (d) incongruent monocular and binocular depth cues (i.e., the monocular cue implied the sentence loomed out of the screen and the binocular cue implied it receded behind the screen). Reading efficiency was mostly unaffected, suggesting that reading in three dimensions is similar to reading in two dimensions. Importantly, fixation disparity was driven by retinal disparity; fixations were significantly more crossed as readers progressed through the sentence in the congruent condition and significantly more uncrossed in the incongruent condition. We conclude that disparity depth cues are used on-line to drive binocular coordination during reading.     

The stereoscopic anisotropy affects manual pointing

Although binocular disparity can in principle provide absolute depth information, perceived stereoscopic depth depends on the relative disparities between points and their spatial arrangement. An example of this is the stereoscopic anisotropy--observers typically perceive less depth for stereoscopic surfaces when depth varies in the horizontal direction than in the vertical direction. We investigated whether this anisotropy also affects manual pointing. Participants were presented with stereograms depicting surfaces that were slanted in depth about either a horizontal axis (inclination) or a vertical axis (slant), and were asked either to point to the edge of a surface, or to estimate its inclination or slant. For both tasks, a clear anisotropy was observed, with participants perceiving greater depth, and also pointing out steeper surfaces, for inclined surfaces than for slanted surfaces. We conclude that both perception and the control of action are subject to a similar stereoscopic anisotropy, and that performance on the two tasks relies on similar depth processing mechanisms.     

Effects of cortical damage on binocular depth perception

Stereoscopic depth perception requires considerable neural computation, including the initial correspondence of the two retinal images, comparison across the local regions of the visual field and integration with other cues to depth. The most common cause for loss of stereoscopic vision is amblyopia, in which one eye has failed to form an adequate input to the visual cortex, usually due to strabismus (deviating eye) or anisometropia. However, the significant cortical processing required to produce the percept of depth means that, even when the retinal input is intact from both eyes, brain damage or dysfunction can interfere with stereoscopic vision. In this review, I examine the evidence for impairment of binocular vision and depth perception that can result from insults to the brain, including both discrete damage, temporal lobectomy and more systemic diseases such as posterior cortical atrophy.This article is part of the themed issue ‘Vision in our three-dimensional world’.     

Stereoscopic Offset Makes Objects Easier to Recognize

Binocular vision is obviously useful for depth perception, but it might also enhance other components of visual processing, such as image segmentation. We used naturalistic images to determine whether giving an object a stereoscopic offset of 15-120 arcmin of crossed disparity relative to its background would make the object easier to recognize in briefly presented (33-133 ms), temporally masked displays. Disparity had a beneficial effect across a wide range of disparities and display durations. Most of this benefit occurred whether or not the stereoscopic contour agreed with the object’s luminance contour. We attribute this benefit to an orienting of spatial attention that selected the object and its local background for enhanced 2D pattern processing. At longer display durations, contour agreement provided an additional benefit, and a separate experiment using random-dot stimuli confirmed that stereoscopic contours plausibly contributed to recognition at the longer display durations in our experiment. We conclude that in real-world situations binocular vision confers an advantage not only for depth perception, but also for recognizing objects from their luminance patterns and bounding contours.     

Vertical disparity, egocentric distance and stereoscopic depth constancy: a new interpretation

There has long been a problem concerning the presence in the visual cortex of binocularly activated cells that are selective for vertical stimulus disparities because it is generally believed that only horizontal disparities contribute to stereoscopic depth perception. The accepted view is that stereoscopic depth estimates are only relative to the fixation point and that independent information from an extraretinal source is needed to scale for absolute or egocentric distance. Recently, however, theoretical computations have shown that egocentric distance can be estimated directly from vertical disparities without recourse to extraretinal sources. There has been little impetus to follow up these computations with experimental observations, because the vertical disparities that normally occur between the images in the two eyes have always been regarded as being too small to be of significance for visual perception and because experiments have consistently shown that our conscious appreciation of egocentric distance is rather crude and unreliable. Nevertheless, the veridicality of stereoscopic depth constancy indicates that accurate distance information is available to the visual system and that the information about egocentric distance and horizontal disparity are processed together so as to continually recalibrate the horizontal disparity values for different absolute distances. Computations show that the recalibration can be based directly on vertical disparities without the need for any intervening estimates of absolute distance. This may partly explain the relative crudity of our conscious appreciation of egocentric distance. From published data it has been possible to calculate the magnitude of the vertical disparities that the human visual system must be able to discriminate in order for depth constancy to have the observed level of veridicality. From published data on the induced effect it has also been possible to calculate the threshold values for the detection of vertical disparities by the visual system. These threshold values are smaller than those needed to provide for the recalibration of the horizontal disparities in the interests of veridical depth constancy. An outline is given of the known properties of the binocularly activated cells in the striate cortex that are able to discriminate and assess the vertical disparities. Experiments are proposed that should validate, or otherwise, the concepts put forward in this paper.     

Stereo vision by self-organization

We propose a new algorithm for stereoscopic depth perception, where the depth map is the momentary state of a dynamic process. To each image point we assign a set of possible disparity values. In a dynamic process with competition and cooperation, the correct disparity value is selected for each image point. Therefore, we solve the correspondence problem by a dynamic, self-organizing process, the structure of which shows analogies to the human visual system. The algorithm can be implemented in a massive parallel manner and yields good results for either artificial or natural images. Received: 1 July 1993/Accepted in revised form: 22 December 1993     

Factors contributing to depth perception: behavioral studies on the reverse perspective illusion

In three behavioral experiments using depth-inverted visual stimuli, the factors that contribute to the 'reverse perspective' illusion were measured. The density of linear perspective grid lines was found to induce the illusion most strongly, followed by shading/shadows, and texture/color information. The relative contributions of such pictorial cues to depth perception are similar to those that facilitate the normal perception of 3D space in 2D paintings.     

Stereopsis activates V3A and caudal intraparietal areas in macaques and humans

Stereopsis, the perception of depth from small differences between the images in the two eyes, provides a rich model for investigating the cortical construction of surfaces and space. Although disparity-tuned cells have been found in a large number of areas in macaque visual cortex, stereoscopic processing in these areas has never been systematically compared using the same stimuli and analysis methods. In order to examine the global architecture of stereoscopic processing in primate visual cortex, we studied fMRI activity in alert, fixating human and macaque subjects. In macaques, we found strongest activation to near/far compared to zero disparity in areas V3, V3A, and CIPS. In humans, we found strongest activation to the same stimuli in areas V3A, V7, the V4d topolog (V4d-topo), and a caudal parietal disparity region (CPDR). Thus, in both primate species a small cluster of areas at the parieto-occipital junction appears to be specialized for stereopsis.