Depth Perception Not Found in Human Observers for Static or Dynamic Anti-Correlated Random Dot Stereograms

One of the greatest challenges in visual neuroscience is that of linking neural activity with perceptual experience. In the case of binocular depth perception, important insights have been achieved through comparing neural responses and the perception of depth, for carefully selected stimuli. One of the most important types of stimulus that has been used here is the anti-correlated random dot stereogram (ACRDS). In these stimuli, the contrast polarity of one half of a stereoscopic image is reversed. While neurons in cortical area V1 respond reliably to the binocular disparities in ACRDS, they do not create a sensation of depth. This discrepancy has been used to argue that depth perception must rely on neural activity elsewhere in the brain. Currently, the psychophysical results on which this argument rests are not clear-cut. While it is generally assumed that ACRDS do not support the perception of depth, some studies have reported that some people, some of the time, perceive depth in some types of these stimuli. Given the importance of these results for understanding the neural correlates of stereopsis, we studied depth perception in ACRDS using a large number of observers, in order to provide an unambiguous conclusion about the extent to which these stimuli support the perception of depth. We presented observers with random dot stereograms in which correlated dots were presented in a surrounding annulus and correlated or anti-correlated dots were presented in a central circular region. While observers could reliably report the depth of the central region for correlated stimuli, we found no evidence for depth perception in static or dynamic anti-correlated stimuli. Confidence ratings for stereoscopic perception were uniformly low for anti-correlated stimuli, but showed normal variation with disparity for correlated stimuli. These results establish that the inability of observers to perceive depth in ACRDS is a robust phenomenon.     

Interaction between pools of binocular disparity detectors tuned to different disparities

Unambiguous dots (having one binocular disparity) when inserted in an ambiguous random-dot stereogram (with multiple disparity values) could pull the ambiguous percept. The unambiguous bias carried that ambiguous depth percept whose disparity was nearest to the disparity of the bias. The closer the disparities were to each other, the stronger the pulling effect that was observed. Even a physical bias of 4% density was adequate to overcome the natural bias of most observers. The stimulus duration had to be over 50 msec to provide a strong pulling effect. In all experiments the stimulus duration was 160 msec or shorter, indicating that the pulling effect was a product of neural interactions, rather than convergence movement of the eyes. As a result of these findings a parallel model of stereopsis has been proposed, which extends the spring-coupled dipole model of Julesz (1971).     

A scheme for binocular depth perception suggested by neurophysiological evidence

A scheme suggested by neurophysiological evidence is proposed to account for the perceptual phenomena related to binocular stereopsis, especially those observed with Julesz' random stereograms. In the scheme, monocular local features are extracted first. Then the correspondence between the left and right local features is searched for. The correspondence is not one-to-one in general. Thus a sort of direction column due to Blakemore is formed. Each unit in the column is binocular and the receptive field belonging to one eye is located in the same part of the visual field as long as the unit belongs to the same column. However, the receptive fields belonging to the other eye are horizontally displaced to one another. That is, each unit is characterized by binocular disparity. If the correspondence is not one-to-one, then several units belonging to the same column respond simultaneously. Binocular stereopsis can be established if one-to-one correspondence is determined to yield global three dimensional regions. The determination of one-to-one correspondence is carried out through a sort of laterally interacting circuitry in the disparity domain. After the determination of local correspondence, three dimensional global regions are formed by detecting the boundary and by filling-in occluded regions. The results of computer simulation are presented regarding Julesz' stereograms with various types of perturbation. Furthermore, the case of random-dot stereogram in which there is a size difference between the left and right images is simulated. Finally, the computer simulation related to the hysteresis in binocular depth perception is carried out.     

Binocular single vision and perceptual processing.

Stimuli with small binocular disparities are seen as single, despite their differing visual directions for the two eyes. Such stimuli also yield stereopsis, but stereopsis and single vision can be dissociated. The occurrence of binocular single vision depends not only on the disparities of individual stimulus elements, but also on the geometrical relation of different parts of the pattern presented to each eye. A pair of vertical bars with opposite binocular disparities is seen as single if the pair is moderately widely spaced but not if it is narrow. Vertical alignment and identity in length of such bars also increase the occurrence of double vision. It is argued that these effects reflect the extraction of features of the monocular patterns, with these detected monocular features determining the binocular percept. Single and double vision of bars differing in orientation can be similarly analysed. The occurrence of relatively elaborate processing of monocular signals does not exclude the possibility that binocular interaction can occur between signals that have not been so processed. Multiple sites or types of binocular interaction are likely.     

Cue combination in the motion correspondence problem

Image motion is a primary source of visual information about the world. However, before this information can be used the visual system must determine the spatio-temporal displacements of the features in the dynamic retinal image, which originate from objects moving in space. This is known as the motion correspondence problem. We investigated whether cross-cue matching constraints contribute to the solution of this problem, which would be consistent with physiological reports that many directionally selective cells in the visual cortex also respond to additional visual cues. We measured the maximum displacement limit (Dmax) for two-frame apparent motion sequences. Dmax increases as the number of elements in such sequences decreases. However, in our displays the total number of elements was kept constant while the number of a subset of elements, defined by a difference in contrast polarity, binocular disparity or colour, was varied. Dmax increased as the number of elements distinguished by a particular cue was decreased. Dmax was affected by contrast polarity for all observers, but only some observers were influenced by binocular disparity and others by colour information. These results demonstrate that the human visual system exploits local, cross-cue matching constraints in the solution of the motion correspondence problem.     

Human stereovision without localized image features

Many theories of human stereovision are based on feature matching and the related correspondence problem. In this paper, we present psychophysical experiments indicating that localized image features such as Laplacian zerocrossings, intensity extrema, or centroids are not necessary for binocular depth perception. Smooth one-dimensional intensity profiles were combined into stereograms with mirror-symmetric half-images such that these localized image features were either absent or did not carry stereo information. In a discrimination task, subjects were asked to distinguish between stereograms differing only by an exchange of these half-images (ortho- vs. pseudoscopic stereograms). In a depth ordering task, subjects had to judge which of the two versions appeared in front. Subjects are able to solve both tasks even in the absence of the mentioned image features. The performance is compared to various possible stereo mechanisms. We conclude that localized image features and the correspondences between them are not necessary to perceive stereoscopic depth. One mechanism accounting for our data is correlation or mean square difference. Received: 8 February 1994 / Accepted in revised form: 15 September 1994     

Binocular fusion in Panum''''s limiting case of stereopsis obeys the uniqueness constraint

In the information processing procedure of stereo vision, the uniqueness constraint has been used as one of the constraints to solve the "correspondence problem". While the uniqueness constraint is valid in most cases, whether it is still valid in some particular stimulus configuration (such as Panum's limiting case) has been a problem of widespread debate for a long time. To investigate the problem, we adopted the Panum's limiting case as its basic stimulus configuration, and delved into the phenomenon of binocular fusion from two distinct aspects: visual direction and orientation disparity. The results show that in Panum's limiting case binocular fusion does not comply with the rules governing regular binocular fusion as far as visual direction and orientation disparity are concerned. This indicates that double fusion does not happen in Panum's limiting case and that the uniqueness constraint is still valid.     

A model of binocular stereopsis including a global consistency constraint

 The binocular correspondence problem was solved by implementing the uniqueness constraint and the continuity constraint, as proposed by Marr and Poggio [Marr D, PoggioT (1976) Science 194: 283–287]. However, these constraints are not sufficient to define the proper correspondence uniquely. With these constraints, random-dot stereograms (RDSs), consisting of the periodic textures in each image, are treated as a correspondence of surfaces composed of patches of alternating values of disparity. This is quite different from the surface we perceive through the RDSs, that is a surface characterized by a single depth. Because these constraints are local, they cannot produce the global optimum of correspondence. To obtain the global optimum of correspondence, we propose a model of binocular stereopsis in which a global measure of correspondence is explicitly employed. The model consists of two hierarchical systems. First, the lower system processes various correspondences based on the uniqueness constraint. Second, the higher system provides a global measure of correspondence for the disparity in question. The higher system uniquely determines the global optimum of correspondence in the lower system through the recurrent loop between hierarchical systems. The convergence of the recurrent loop is determined by the consistency between the hierarchical systems. The condition is termed the `global consistency constraint. Received: 27 August 1998 / Accepted in revised form: 8 November 1999     

The stereoscopic anisotropy affects manual pointing

Although binocular disparity can in principle provide absolute depth information, perceived stereoscopic depth depends on the relative disparities between points and their spatial arrangement. An example of this is the stereoscopic anisotropy--observers typically perceive less depth for stereoscopic surfaces when depth varies in the horizontal direction than in the vertical direction. We investigated whether this anisotropy also affects manual pointing. Participants were presented with stereograms depicting surfaces that were slanted in depth about either a horizontal axis (inclination) or a vertical axis (slant), and were asked either to point to the edge of a surface, or to estimate its inclination or slant. For both tasks, a clear anisotropy was observed, with participants perceiving greater depth, and also pointing out steeper surfaces, for inclined surfaces than for slanted surfaces. We conclude that both perception and the control of action are subject to a similar stereoscopic anisotropy, and that performance on the two tasks relies on similar depth processing mechanisms.     

Stereoscopic Vision in the Absence of the Lateral Occipital Cortex

Both dorsal and ventral cortical visual streams contain neurons sensitive to binocular disparities, but the two streams may underlie different aspects of stereoscopic vision. Here we investigate stereopsis in the neurological patient D.F., whose ventral stream, specifically lateral occipital cortex, has been damaged bilaterally, causing profound visual form agnosia. Despite her severe damage to cortical visual areas, we report that DF''s stereo vision is strikingly unimpaired. She is better than many control observers at using binocular disparity to judge whether an isolated object appears near or far, and to resolve ambiguous structure-from-motion. DF is, however, poor at using relative disparity between features at different locations across the visual field. This may stem from a difficulty in identifying the surface boundaries where relative disparity is available. We suggest that the ventral processing stream may play a critical role in enabling healthy observers to extract fine depth information from relative disparities within one surface or between surfaces located in different parts of the visual field.     

Binocular fusion in Panum’s limiting case of stereopsis obeys the uniqueness constraint

In the information processing procedure of stereo vision, the uniqueness constraint has been used as one of the constraints to solve the “correspondence problem”. While the uniqueness constraint is valid in most cases, whether it is still valid in some particular stimulus configuration (such as Panum’s limiting case) has been a problem of widespread debate for a long time. To investigate the problem, we adopted the Panum’s limiting case as its basic stimulus configuration, and delved into the phenomenon of binocular fusion from two distinct aspects: visual direction and orientation disparity. The results show that in Panum’s limiting case binocular fusion does not comply with the rules governing regular binocular fusion as far as visual direction and orientation disparity are concerned. This indicates that double fusion does not happen in Panum’s limiting case and that the uniqueness constraint is still valid.     

Binocular fusion in Panum’s limiting case of stereopsis obeys the uniqueness constraint

In the information processing procedure of stereo vision, the uniqueness constraint has been used as one of the constraints to solve the “correspondence problem”. While the uniqueness constraint is valid in most cases, whether it is still valid in some particular stimulus configuration (such as Panum’s limiting case) has been a problem of widespread debate for a long time. To investigate the problem, we adopted the Panum’s limiting case as its basic stimulus configuration, and delved into the phenomenon of binocular fusion from two distinct aspects: visual direction and orientation disparity. The results show that in Panum’s limiting case binocular fusion does not comply with the rules governing regular binocular fusion as far as visual direction and orientation disparity are concerned. This indicates that double fusion does not happen in Panum’s limiting case and that the uniqueness constraint is still valid.     

Long-distance interactions in Cyclopean vision

We report on evidence for selective long-distance interactions in Cyclopean binocular vision. When presented with a pair of Cyclopean test bars observers could discriminate trial-to-trial uncorrelated variations in the mean orientation, orientation difference, separation and mean location of the test bars while ignoring random variations in the orientation, width and location of a third bar placed between the two test bars. We propose that the human visual system contains Cyclopean long-distance comparators (i) that compare the outputs of two narrow receptive fields some distance apart while being insensitive to stimuli located between those receptive fields, and (ii) the outputs of which carry orthogonally labelled indicators of orientation difference, mean orientation, separation and mean location. In the evolutionary context, one role for the proposed mechanisms might be to encode information about the silhouettes of animals whose camouflage is broken by the binocular vision of predators.     

Visual depth encoding in populations of neurons with localized receptive fields

 Stereopsis is the ability to perceive three-dimensional structure from disparities between the two-dimensional retinal images. Although disparity-sensitive neurons have been proposed as a neural representation of this ability many years ago, it is still difficult to link all qualities of stereopsis to properties of the neural correlate of binocular disparities. The present study wants to support efforts directed at closing the gap between electrophysiology and psychophysics. Populations of disparity-sensitive neurons in V1 were simulated using the energy-neuron model. Responses to different types of stimuli were evaluated with an efficient statistical estimator and related to psychophysical findings. The representation of disparity in simulated population responses appeared to be very robust. Small populations allowed good depth discrimination. Two types of energy neurons (phase- and position-type models) that are discussed as possible neural implementations of disparity-selectivity could be compared to each other. Phase-type coding was more robust and could explain a tendency towards zero disparity in degenerated stimuli and, for high-pass stimuli, exhibited the breakdown of disparity discrimination at a maximum disparity value. Contrast-inverted stereograms led to high variances in disparity representation, which is a possible explanation of the absence of depth percepts in large contrast-inverted stimuli. Our study suggests that nonlocal interactions destroy depth percepts in large contrast-inverted stereograms, although these percepts occur for smaller stimuli of the same class. Received: 21 December 2001 / Accepted: 29 April 2002 RID="*" ID="*" Present address: Bayer AG BTS-PT-MVT-MKM, Geb. K9, 51368 Leverkusen, Germany Acknowledgement. This work was supported by a scholarship from the Studienstiftung des deutschen Volkes to J.L. Correspondence to: J. Lippert (e-mail: joerg.lippert.jl@bayer-ag.de)     

Stereopsis and the random element in the organization of the striate cortex.

Receptive field position and orientation disparities are both properties of binocularly discharged striate neurons. Receptive field position desparities have been used as a key element in the neural theory for binocular depth discrimination. Since most striate cells in the cat are binocular, these position disparities require that cells immediately adjacent to one another in the cortex should show a random scatter in their monocular receptive field positions. Superimposed on the progressive topographical representation of the visual field on the striate cortex there is experimental evidence for a localized monocular receptive field position scatter. The suggestion is examined that the binocular position disparities are built up out of the two monocular position scatters. An examination of receptive field orientation disparities and their relation to the random variation in the monocular preferred orientations of immediately adjacent striate neurons also leads to the conclusion that binocular orientation disparities are a consequence of the two monocular scatters. As for receptive field position, the local scatter in preferred orientation is superimposed on a progressive representation of orientation over larger areas of the cortex. The representation in the striate cortex of visual field position and of stimulus orientation is examined in relation to the correlation between the disparities in receptive field position and preferred orientation. The role of orientation disparities in binocular vision is reviewed.     

The role of vergence in the perception of distance: a fair test of Bishop Berkeley's claim

Binocular eye movements were measured while subjects perceived the wallpaper illusion in order to test the claim made by Bishop Berkeley in 1709 that we perceive the distance of nearby objects by evaluating the vergence angles of our eyes. Four subjects looked through a nearby fronto-parallel array of vertical rods (28-35 cm away) as they binocularly fixated a point about 1 meter away. The wallpaper illusion was perceived under these conditions, i.e. the rods appeared farther away than their physical location. We found that although binocular fixation at an appropriate distance was needed to begin perceiving the wallpaper illusion (at least for naive observers), once established, the illusion was quite robust in the sense that it was not affected by changing vergence. No connection between the apparent localization of the rods and vergence was observed. We conclude that it is unlikely that vergence, itself, is responsible for the perceived distance shift in the wallpaper illusion, making it unlikely that vergence contributes to the perception of distance as Bishop Berkeley suggested. We found this to be true even when vergence angles were relatively large (more than 2 deg), the region in which the control of vergence eye movements has been shown to be both fast and effective.     

Evidence for surface-based processing of binocular disparity

It is convenient to think of an object's location as a point within a Cartesian framework; the x axis corresponds to right and left, the y axis to up and down, and the z axis to forward or backward. When an observer is looking straight ahead, binocular disparities provide information about distance along the z axis from the fixation plane. In this coordinate system, changes in disparity are treated as independent of changes in location along the orthogonal x and y axes. Does the human visual system use this three-dimensional coordinate system, or does it specify feature location in a coordinate frame determined by other nearby visible features? Here we show that the sensitivity of the human stereo system is determined by the distance of points with respect to a local reference plane, rather than by the distance along the z axis with respect to the fixation plane. There is a distinct advantage to using a local frame of reference for specifying location. It obviates the need to construct a complex three-dimensional space in either eye-centered or head-centered coordinates that must be updated with every shift of the eyes and head.     

Prey orienting in frogs: Accounting for variations in output with stimulus distance

Summary We have studied the responses of leopard frogs,Rana pipiens, to live mealworms presented at different distances on the mid-sagittal plane. The response of normal frogs to stimuli at nearer distances consists of a direct snap whose amplitude increases with stimulus distance. For greater distances, the response consists of a forward hop whose amplitude also varies with stimulus distance. Over an intermediate range of distances, responses may be either snaps or hops. Whichever response occurs is of appropriate amplitude. The distance at which frogs switch from predominantly snapping responses to predominantly hopping responses increases with body size.Like normal frogs, unilaterally blinded frogs respond to stimuli at nearer distances with snaps whose amplitude varies with stimulus distance, switch from snapping to hopping over an intermediate range of distances, and respond to stimuli at greater distances with hops whose amplitude also increases with stimulus distance. In many cases, unilateral blinding did however result in a decrease in the distance at which the frogs switched from snapping to hopping. Such changes were not accompanied by the changes in snap or hop amplitude which would be expected if unilateral blinding resulted in generalized changes in distance judgement. Normal variations in snap amplitude and switches from snapping to hopping were also observed in frogs subjected to unilateral eye removal prior to the metamorphic eye migration which creates the adult binocular visual field.These results imply that neither distance discrimination nor any of the kinds of variation in motor output which occur with increasing stimulus distance necessarily depend on binocular cues. The behaviors studied also appear to be largely independent of normal binocular experience. More generally, our results suggest that the movement triggered by a stimulus at a particular location is not determined entirely by the retinal and superficial tectal region activated but rather reflects a combination of a retinal local sign signal with other kinds of information. The latter probably include signals related to stimulus distance and body posture, and may include signals related to body size as well.     

A laminar cortical model of stereopsis and 3D surface perception: closure and da Vinci stereopsis

A laminar cortical model of stereopsis and 3D surface perception is developed and simulated. The model describes how monocular and binocular oriented filtering interact with later stages of 3D boundary formation and surface filling-in in the LGN and cortical areas V1, V2, and V4. It proposes how interactions between layers 4, 3B, and 2/3 in V1 and V2 contribute to stereopsis, and how binocular and monocular information combine to form 3D boundary and surface representations. The model includes two main new developments: (1) It clarifies how surface-to-boundary feedback from V2 thin stripes to pale stripes helps to explain data about stereopsis. This feedback has previously been used to explain data about 3D figure-ground perception. (2) It proposes that the binocular false match problem is subsumed under the Gestalt grouping problem. In particular, the disparity filter, which helps to solve the correspondence problem by eliminating false matches, is realized using inhibitory interneurons as part of the perceptual grouping process by horizontal connections in layer 2/3 of cortical area V2. The enhanced model explains all the psychophysical data previously simulated by Grossberg and Howe (2003), such as contrast variations of dichoptic masking and the correspondence problem, the effect of interocular contrast differences on stereoacuity, Panum's limiting case, the Venetian blind illusion, stereopsis with polarity-reversed stereograms, and da Vinci stereopsis. It also explains psychophysical data about perceptual closure and variations of da Vinci stereopsis that previous models cannot yet explain.     

Stereoscopic acuity and observation distance

The amount of depth perceived from a fixed pattern of horizontal disparities varies with viewing distance. We investigated whether thresholds for discriminating stereoscopic corrugations at a range of spatial frequencies were also affected by viewing distance or whether they were determined solely by the angular disparity in the stimulus prior to scaling. Although thresholds were found to be determined primarily by disparity over a broad range of viewing distances, they were on average a factor of two higher at the shortest viewing distance (28.5 cm) than at larger viewing distances (57 to 450 cm). We found the same pattern of results when subjects' accommodation was arranged to be the same at all viewing distances. The change in thresholds at close distances is in the direction expected if subjects' performance is limited by a minimum perceived depth.