The direction of retinal motion facilitates binocular stereopsis.

Visual information from binocular disparity and from relative motion provide information about three-dimensional structure and layout of the world. Although the mechanisms that process these cues have typically been studied independently, there is now a substantial body of evidence that suggests that they interact in the visual pathway. This paper investigates one advantage of such an interaction: whether retinal motion can be used as a matching constraint in the binocular correspondence process. Stimuli that contained identical disparity and motion signals but which differed in their fine-scale correlation were created to establish whether the direction, or the speed, of motion could enhance performance in a psychophysical task in which binocular matching is a limiting factor. The results of these experiments provide clear evidence that different directions of motion, but not different speeds, are processed separately in stereopsis. The results fit well with properties of neurons early in the cortical visual pathway which are thought to be involved in determining local matches between features in the two eyes'' images.     

Binocular fusion in Panum''''s limiting case of stereopsis obeys the uniqueness constraint

In the information processing procedure of stereo vision, the uniqueness constraint has been used as one of the constraints to solve the "correspondence problem". While the uniqueness constraint is valid in most cases, whether it is still valid in some particular stimulus configuration (such as Panum's limiting case) has been a problem of widespread debate for a long time. To investigate the problem, we adopted the Panum's limiting case as its basic stimulus configuration, and delved into the phenomenon of binocular fusion from two distinct aspects: visual direction and orientation disparity. The results show that in Panum's limiting case binocular fusion does not comply with the rules governing regular binocular fusion as far as visual direction and orientation disparity are concerned. This indicates that double fusion does not happen in Panum's limiting case and that the uniqueness constraint is still valid.     

On the inverse problem of binocular 3D motion perception

It is shown that existing processing schemes of 3D motion perception such as interocular velocity difference, changing disparity over time, as well as joint encoding of motion and disparity, do not offer a general solution to the inverse optics problem of local binocular 3D motion. Instead we suggest that local velocity constraints in combination with binocular disparity and other depth cues provide a more flexible framework for the solution of the inverse problem. In the context of the aperture problem we derive predictions from two plausible default strategies: (1) the vector normal prefers slow motion in 3D whereas (2) the cyclopean average is based on slow motion in 2D. Predicting perceived motion directions for ambiguous line motion provides an opportunity to distinguish between these strategies of 3D motion processing. Our theoretical results suggest that velocity constraints and disparity from feature tracking are needed to solve the inverse problem of 3D motion perception. It seems plausible that motion and disparity input is processed in parallel and integrated late in the visual processing hierarchy.     

Binocular fusion in Panum’s limiting case of stereopsis obeys the uniqueness constraint

In the information processing procedure of stereo vision, the uniqueness constraint has been used as one of the constraints to solve the “correspondence problem”. While the uniqueness constraint is valid in most cases, whether it is still valid in some particular stimulus configuration (such as Panum’s limiting case) has been a problem of widespread debate for a long time. To investigate the problem, we adopted the Panum’s limiting case as its basic stimulus configuration, and delved into the phenomenon of binocular fusion from two distinct aspects: visual direction and orientation disparity. The results show that in Panum’s limiting case binocular fusion does not comply with the rules governing regular binocular fusion as far as visual direction and orientation disparity are concerned. This indicates that double fusion does not happen in Panum’s limiting case and that the uniqueness constraint is still valid.     

Binocular fusion in Panum’s limiting case of stereopsis obeys the uniqueness constraint

In the information processing procedure of stereo vision, the uniqueness constraint has been used as one of the constraints to solve the “correspondence problem”. While the uniqueness constraint is valid in most cases, whether it is still valid in some particular stimulus configuration (such as Panum’s limiting case) has been a problem of widespread debate for a long time. To investigate the problem, we adopted the Panum’s limiting case as its basic stimulus configuration, and delved into the phenomenon of binocular fusion from two distinct aspects: visual direction and orientation disparity. The results show that in Panum’s limiting case binocular fusion does not comply with the rules governing regular binocular fusion as far as visual direction and orientation disparity are concerned. This indicates that double fusion does not happen in Panum’s limiting case and that the uniqueness constraint is still valid.     

Assessment of novel binocular colour, motion and contrast tests in glaucoma

The effects of glaucoma on binocular visual sensitivity for the detection of various stimulus attributes are investigated at the fovea and in four paracentral retinal regions. The study employed a number of visual stimuli designed to isolate the processing of various stimulus attributes. We measured absolute contrast detection thresholds and functional contrast sensitivity by using Landolt ring stimuli. This psychophysical Landolt C-based contrast test of detection and gap discrimination allowed us to test parafoveally at 6 ° from fixation and foveally by employing interleaved testing locations. First-order motion perception was examined by using moving stimuli embedded in static luminance contrast noise. Red/green (RG) and yellow/blue (YB) colour thresholds were measured with the Colour Assessment and Diagnosis (CAD) test, which utilises random dynamic luminance contrast noise (± 45 %) to ensure that only colour and not luminance signals are available for target detection. Subjects were normal controls (n?=?65) and glaucoma patients with binocular visual field defects (n?=?15) classified based on their Humphrey Field Analyzer mean deviation (MD) scores. The impairment of visual function varied depending on the stimulus attribute and location tested. Progression of loss was noted for all tests as the degree of glaucoma increased. For subjects with mild glaucoma (MD ?0.01 dB to ?6.00 dB) significantly more data points fell outside the normal age-representative range for RG colour thresholds than for any other visual test, followed by motion thresholds. This was particularly the case for the parafoveal data compared with the foveal data. Thus, a multifaceted measure of binocular visual performance, incorporating RG colour and motion test at multiple locations, might provide a better index for comparison with quality of life measures in glaucoma.     

Shading Beats Binocular Disparity in Depth from Luminance Gradients: Evidence against a Maximum Likelihood Principle for Cue Combination

Perceived depth is conveyed by multiple cues, including binocular disparity and luminance shading. Depth perception from luminance shading information depends on the perceptual assumption for the incident light, which has been shown to default to a diffuse illumination assumption. We focus on the case of sinusoidally corrugated surfaces to ask how shading and disparity cues combine defined by the joint luminance gradients and intrinsic disparity modulation that would occur in viewing the physical corrugation of a uniform surface under diffuse illumination. Such surfaces were simulated with a sinusoidal luminance modulation (0.26 or 1.8 cy/deg, contrast 20%-80%) modulated either in-phase or in opposite phase with a sinusoidal disparity of the same corrugation frequency, with disparity amplitudes ranging from 0’-20’. The observers’ task was to adjust the binocular disparity of a comparison random-dot stereogram surface to match the perceived depth of the joint luminance/disparity-modulated corrugation target. Regardless of target spatial frequency, the perceived target depth increased with the luminance contrast and depended on luminance phase but was largely unaffected by the luminance disparity modulation. These results validate the idea that human observers can use the diffuse illumination assumption to perceive depth from luminance gradients alone without making an assumption of light direction. For depth judgments with combined cues, the observers gave much greater weighting to the luminance shading than to the disparity modulation of the targets. The results were not well-fit by a Bayesian cue-combination model weighted in proportion to the variance of the measurements for each cue in isolation. Instead, they suggest that the visual system uses disjunctive mechanisms to process these two types of information rather than combining them according to their likelihood ratios.     

A model of binocular stereopsis including a global consistency constraint

 The binocular correspondence problem was solved by implementing the uniqueness constraint and the continuity constraint, as proposed by Marr and Poggio [Marr D, PoggioT (1976) Science 194: 283–287]. However, these constraints are not sufficient to define the proper correspondence uniquely. With these constraints, random-dot stereograms (RDSs), consisting of the periodic textures in each image, are treated as a correspondence of surfaces composed of patches of alternating values of disparity. This is quite different from the surface we perceive through the RDSs, that is a surface characterized by a single depth. Because these constraints are local, they cannot produce the global optimum of correspondence. To obtain the global optimum of correspondence, we propose a model of binocular stereopsis in which a global measure of correspondence is explicitly employed. The model consists of two hierarchical systems. First, the lower system processes various correspondences based on the uniqueness constraint. Second, the higher system provides a global measure of correspondence for the disparity in question. The higher system uniquely determines the global optimum of correspondence in the lower system through the recurrent loop between hierarchical systems. The convergence of the recurrent loop is determined by the consistency between the hierarchical systems. The condition is termed the `global consistency constraint. Received: 27 August 1998 / Accepted in revised form: 8 November 1999     

Luminance,Colour, Viewpoint and Border Enhanced Disparity Energy Model

The visual cortex is able to extract disparity information through the use of binocular cells. This process is reflected by the Disparity Energy Model, which describes the role and functioning of simple and complex binocular neuron populations, and how they are able to extract disparity. This model uses explicit cell parameters to mathematically determine preferred cell disparities, like spatial frequencies, orientations, binocular phases and receptive field positions. However, the brain cannot access such explicit cell parameters; it must rely on cell responses. In this article, we implemented a trained binocular neuronal population, which encodes disparity information implicitly. This allows the population to learn how to decode disparities, in a similar way to how our visual system could have developed this ability during evolution. At the same time, responses of monocular simple and complex cells can also encode line and edge information, which is useful for refining disparities at object borders. The brain should then be able, starting from a low-level disparity draft, to integrate all information, including colour and viewpoint perspective, in order to propagate better estimates to higher cortical areas.     

Effects of pattern element density upon displacement limits for motion detection in random binary luminance patterns.

The upper motion displacement threshold (Dmax) was determined with two-frame motion sequences of random binary luminance patterns, over a range of pattern element sizes and densities. Dmax was little affected by density at small element sizes (less than 5 arcmin), in agreement with previous reports. However, at larger element sizes (greater than 9 arcmin) Dmax increased as element density was reduced in the range 50-5%. We explain our findings by a model which takes into account spatial-frequency filtering prior to motion detection, and the effects of pattern density upon the statistics of random binary patterns. We also implicate the dependence of Dmax upon the contrast energy of the elements in broadband patterns, and provide a direct demonstration that Dmax is contrast limited over a wide range of pattern contrasts (72-2.5%). Previous reports that Dmax is independent of density should be modified to take into account the complex effects of density upon the statistics of random patterns, and the existence of physiological filtering prior to motion detection.     

Stereoscopic Vision in the Absence of the Lateral Occipital Cortex

Both dorsal and ventral cortical visual streams contain neurons sensitive to binocular disparities, but the two streams may underlie different aspects of stereoscopic vision. Here we investigate stereopsis in the neurological patient D.F., whose ventral stream, specifically lateral occipital cortex, has been damaged bilaterally, causing profound visual form agnosia. Despite her severe damage to cortical visual areas, we report that DF''s stereo vision is strikingly unimpaired. She is better than many control observers at using binocular disparity to judge whether an isolated object appears near or far, and to resolve ambiguous structure-from-motion. DF is, however, poor at using relative disparity between features at different locations across the visual field. This may stem from a difficulty in identifying the surface boundaries where relative disparity is available. We suggest that the ventral processing stream may play a critical role in enabling healthy observers to extract fine depth information from relative disparities within one surface or between surfaces located in different parts of the visual field.     

Stereoscopic correspondence for ambiguous targets is affected by elevation and fixation distance

Binocular correspondence must be determined if disparity is to be used to provide information about three-dimensional shape. The current study investigated whether knowledge of the statistical distribution of disparities in the natural environment is employed in this process. A simple model, which produces distributions of distances similar to those found in the natural environment, was used to predict the distribution of disparities in natural images. This model predicts that crossed disparities will be more likely as (i) stimulus elevation decreases below fixation and (ii) fixation distance increases. To determine whether these factors influence binocular correspondence for human observers, ambiguous stereograms were presented to observers, as stimulus elevation and fixation distance were manipulated. Clear biases were observed in the depth perceived in these stereograms, which were more likely to be seen as closer than fixation (i) for stimuli presented below fixation and (ii) as fixation distance increased. These results suggest that binocular correspondence is determined in a manner consistent with the distributions of disparities expected in natural scenes.     

Blur and disparity are complementary cues to depth

Estimating depth from binocular disparity is extremely precise, and the cue does not depend on statistical regularities in the environment. Thus, disparity is commonly regarded as the best visual cue for determining 3D layout. But depth from disparity is only precise near where one is looking; it is quite imprecise elsewhere. Away from fixation, vision resorts to using other depth cues-e.g., linear perspective, familiar size, aerial perspective. But those cues depend on statistical regularities in the environment and are therefore not always reliable. Depth from defocus blur relies on fewer assumptions and has the same geometric constraints as disparity but different physiological constraints. Blur could in principle fill in the parts of visual space where disparity is imprecise. We tested this possibility with a depth-discrimination experiment. Disparity was more precise near fixation and blur was indeed more precise away from fixation. When both cues were available, observers relied on the more informative one. Blur appears to play an important, previously unrecognized role in depth perception. Our findings lead to a new hypothesis about the evolution of slit-shaped pupils and have implications for the design and implementation of stereo 3D displays.     

Size-disparity correlation in human binocular depth perception.

To use the small horizontal disparities between images projected to the eyes for the recovery of three-dimensional information, our visual system must first identify which feature in one eye's image corresponds with which in the other. The earliest level of disparity processing in primates (V1) contains cells that are spatial-frequency tuned. If such cells have a disparity range that covers only a single period of their mean tuning frequency, there will always be exactly one potential match within this range. Here, this 'size-disparity' hypothesis was tested by measuring the contrast sensitivity of stereopsis as a function of disparity for single bandpass-filtered items. It was found that thresholds were low and relatively constant up to disparities an order of magnitude larger than is predicted by this constraint. Furthermore, peak sensitivity was relatively independent of spatial frequency. A control experiment showed that binocular correlation of the carrier is necessary for this task. In a third experiment, the maximum disparity that supports threshold performance was compared for an isolated bandpass item and bandpass-filtered noise. This limit was found to be five times larger for the isolated stimuli. In summary, these findings show that the initial stage of disparity detection is not limited by the size-disparity constraint. For stimuli with multiple false targets, however, processes subsequent to this stage reduce the disparity range over which the correspondence problem can be solved.     

Humans ignore motion and stereo cues in favor of a fictional stable world

As we move through the world, our eyes acquire a sequence of images. The information from this sequence is sufficient to determine the structure of a three-dimensional scene, up to a scale factor determined by the distance that the eyes have moved. Previous evidence shows that the human visual system accounts for the distance the observer has walked and the separation of the eyes when judging the scale, shape, and distance of objects. However, in an immersive virtual-reality environment, observers failed to notice when a scene expanded or contracted, despite having consistent information about scale from both distance walked and binocular vision. This failure led to large errors in judging the size of objects. The pattern of errors cannot be explained by assuming a visual reconstruction of the scene with an incorrect estimate of interocular separation or distance walked. Instead, it is consistent with a Bayesian model of cue integration in which the efficacy of motion and disparity cues is greater at near viewing distances. Our results imply that observers are more willing to adjust their estimate of interocular separation or distance walked than to accept that the scene has changed in size.     

Depth Perception Not Found in Human Observers for Static or Dynamic Anti-Correlated Random Dot Stereograms

One of the greatest challenges in visual neuroscience is that of linking neural activity with perceptual experience. In the case of binocular depth perception, important insights have been achieved through comparing neural responses and the perception of depth, for carefully selected stimuli. One of the most important types of stimulus that has been used here is the anti-correlated random dot stereogram (ACRDS). In these stimuli, the contrast polarity of one half of a stereoscopic image is reversed. While neurons in cortical area V1 respond reliably to the binocular disparities in ACRDS, they do not create a sensation of depth. This discrepancy has been used to argue that depth perception must rely on neural activity elsewhere in the brain. Currently, the psychophysical results on which this argument rests are not clear-cut. While it is generally assumed that ACRDS do not support the perception of depth, some studies have reported that some people, some of the time, perceive depth in some types of these stimuli. Given the importance of these results for understanding the neural correlates of stereopsis, we studied depth perception in ACRDS using a large number of observers, in order to provide an unambiguous conclusion about the extent to which these stimuli support the perception of depth. We presented observers with random dot stereograms in which correlated dots were presented in a surrounding annulus and correlated or anti-correlated dots were presented in a central circular region. While observers could reliably report the depth of the central region for correlated stimuli, we found no evidence for depth perception in static or dynamic anti-correlated stimuli. Confidence ratings for stereoscopic perception were uniformly low for anti-correlated stimuli, but showed normal variation with disparity for correlated stimuli. These results establish that the inability of observers to perceive depth in ACRDS is a robust phenomenon.     

On the Contribution of Binocular Disparity to the Long-Term Memory for Natural Scenes

Binocular disparity is a fundamental dimension defining the input we receive from the visual world, along with luminance and chromaticity. In a memory task involving images of natural scenes we investigate whether binocular disparity enhances long-term visual memory. We found that forest images studied in the presence of disparity for relatively long times (7s) were remembered better as compared to 2D presentation. This enhancement was not evident for other categories of pictures, such as images containing cars and houses, which are mostly identified by the presence of distinctive artifacts rather than by their spatial layout. Evidence from a further experiment indicates that observers do not retain a trace of stereo presentation in long-term memory.     

A Bayesian model of stereopsis depth and motion direction discrimination

 The extraction of stereoscopic depth from retinal disparity, and motion direction from two-frame kinematograms, requires the solution of a correspondence problem. In previous psychophysical work [Read and Eagle (2000) Vision Res 40: 3345–3358], we compared the performance of the human stereopsis and motion systems with correlated and anti-correlated stimuli. We found that, although the two systems performed similarly for narrow-band stimuli, broad-band anti-correlated kinematograms produced a strong perception of reversed motion, whereas the stereograms appeared merely rivalrous. I now model these psychophysical data with a computational model of the correspondence problem based on the known properties of visual cortical cells. Noisy retinal images are filtered through a set of Fourier channels tuned to different spatial frequencies and orientations. Within each channel, a Bayesian analysis incorporating a prior preference for small disparities is used to assess the probability of each possible match. Finally, information from the different channels is combined to arrive at a judgement of stimulus disparity. Each model system – stereopsis and motion – has two free parameters: the amount of noise they are subject to, and the strength of their preference for small disparities. By adjusting these parameters independently for each system, qualitative matches are produced to psychophysical data, for both correlated and anti-correlated stimuli, across a range of spatial frequency and orientation bandwidths. The motion model is found to require much higher noise levels and a weaker preference for small disparities. This makes the motion model more tolerant of poor-quality reverse-direction false matches encountered with anti-correlated stimuli, matching the strong perception of reversed motion that humans experience with these stimuli. In contrast, the lower noise level and tighter prior preference used with the stereopsis model means that it performs close to chance with anti-correlated stimuli, in accordance with human psychophysics. Thus, the key features of the experimental data can be reproduced assuming that the motion system experiences more effective noise than the stereoscopy system and imposes a less stringent preference for small disparities. Received: 2 March 2001 / Accepted in revised form: 5 July 2001     

A laminar cortical model of stereopsis and 3D surface perception: closure and da Vinci stereopsis

A laminar cortical model of stereopsis and 3D surface perception is developed and simulated. The model describes how monocular and binocular oriented filtering interact with later stages of 3D boundary formation and surface filling-in in the LGN and cortical areas V1, V2, and V4. It proposes how interactions between layers 4, 3B, and 2/3 in V1 and V2 contribute to stereopsis, and how binocular and monocular information combine to form 3D boundary and surface representations. The model includes two main new developments: (1) It clarifies how surface-to-boundary feedback from V2 thin stripes to pale stripes helps to explain data about stereopsis. This feedback has previously been used to explain data about 3D figure-ground perception. (2) It proposes that the binocular false match problem is subsumed under the Gestalt grouping problem. In particular, the disparity filter, which helps to solve the correspondence problem by eliminating false matches, is realized using inhibitory interneurons as part of the perceptual grouping process by horizontal connections in layer 2/3 of cortical area V2. The enhanced model explains all the psychophysical data previously simulated by Grossberg and Howe (2003), such as contrast variations of dichoptic masking and the correspondence problem, the effect of interocular contrast differences on stereoacuity, Panum's limiting case, the Venetian blind illusion, stereopsis with polarity-reversed stereograms, and da Vinci stereopsis. It also explains psychophysical data about perceptual closure and variations of da Vinci stereopsis that previous models cannot yet explain.     

The Role of Binocular Disparity in Stereoscopic Images of Objects in the Macaque Anterior Intraparietal Area

Neurons in the macaque Anterior Intraparietal area (AIP) encode depth structure in random-dot stimuli defined by gradients of binocular disparity, but the importance of binocular disparity in real-world objects for AIP neurons is unknown. We investigated the effect of binocular disparity on the responses of AIP neurons to images of real-world objects during passive fixation. We presented stereoscopic images of natural and man-made objects in which the disparity information was congruent or incongruent with disparity gradients present in the real-world objects, and images of the same objects where such gradients were absent. Although more than half of the AIP neurons were significantly affected by binocular disparity, the great majority of AIP neurons remained image selective even in the absence of binocular disparity. AIP neurons tended to prefer stimuli in which the depth information derived from binocular disparity was congruent with the depth information signaled by monocular depth cues, indicating that these monocular depth cues have an influence upon AIP neurons. Finally, in contrast to neurons in the inferior temporal cortex, AIP neurons do not represent images of objects in terms of categories such as animate-inanimate, but utilize representations based upon simple shape features including aspect ratio.