Vertical Binocular Disparity is Encoded Implicitly within a Model Neuronal Population Tuned to Horizontal Disparity and Orientation

Primary visual cortex is often viewed as a “cyclopean retina”, performing the initial encoding of binocular disparities between left and right images. Because the eyes are set apart horizontally in the head, binocular disparities are predominantly horizontal. Yet, especially in the visual periphery, a range of non-zero vertical disparities do occur and can influence perception. It has therefore been assumed that primary visual cortex must contain neurons tuned to a range of vertical disparities. Here, I show that this is not necessarily the case. Many disparity-selective neurons are most sensitive to changes in disparity orthogonal to their preferred orientation. That is, the disparity tuning surfaces, mapping their response to different two-dimensional (2D) disparities, are elongated along the cell''s preferred orientation. Because of this, even if a neuron''s optimal 2D disparity has zero vertical component, the neuron will still respond best to a non-zero vertical disparity when probed with a sub-optimal horizontal disparity. This property can be used to decode 2D disparity, even allowing for realistic levels of neuronal noise. Even if all V1 neurons at a particular retinotopic location are tuned to the expected vertical disparity there (for example, zero at the fovea), the brain could still decode the magnitude and sign of departures from that expected value. This provides an intriguing counter-example to the common wisdom that, in order for a neuronal population to encode a quantity, its members must be tuned to a range of values of that quantity. It demonstrates that populations of disparity-selective neurons encode much richer information than previously appreciated. It suggests a possible strategy for the brain to extract rarely-occurring stimulus values, while concentrating neuronal resources on the most commonly-occurring situations.     

Disparity tuning as simulated by a neural net

Previous research has suggested that the processing of binocular disparity in complex cells may be described with an energy formalism. The energy formalism allows for a representation of disparity by differences in the position or in the phase of monocular receptive subfields of binocular cells, or by combination of these two types. We studied the coding of disparities with an approach complementary to previous algorithmic investigations. Since realization of these representations is probably not genetically determined but learned during ontogeny, we used backpropagation networks to study which of these three possibilities were realized within neural nets. Three types of networks were trained with noise patterns in analogy to the three types of energy models. The networks learned the task and generalized to untrained correlated noise pattern input. Outputs were broadly tuned to spatial frequency and did not respond to anti-correlated noise patterns. Although the energy model was not explicitly implemented, we could analyze the outputs of the networks using predictions of the energy formalism. After learning was completed, the model neurons preferred position shifts over phase shifts in representing disparity. We discuss the general meaning of these findings and the correspondences and deviations between the energy model, V1 neurons, and our networks. Received: 6 August 1999 / Accepted in revised form: 26 January 2000     

Perceptual “Read-Out” of Conjoined Direction and Disparity Maps in Extrastriate Area MT

Cortical neurons are frequently tuned to several stimulus dimensions, and many cortical areas contain intercalated maps of multiple variables. Relatively little is known about how information is “read out” of these multidimensional maps. For example, how does an organism extract information relevant to the task at hand from neurons that are also tuned to other, irrelevant stimulus dimensions? We addressed this question by employing microstimulation techniques to examine the contribution of disparity-tuned neurons in the middle temporal (MT) visual area to performance on a direction discrimination task. Most MT neurons are tuned to both binocular disparity and the direction of stimulus motion, and MT contains topographic maps of both parameters. We assessed the effect of microstimulation on direction judgments after first characterizing the disparity tuning of each stimulation site. Although the disparity of the stimulus was irrelevant to the required task, we found that microstimulation effects were strongly modulated by the disparity tuning of the stimulated neurons. For two of three monkeys, microstimulation of nondisparity-selective sites produced large biases in direction judgments, whereas stimulation of disparity-selective sites had little or no effect. The binocular disparity was optimized for each stimulation site, and our result could not be explained by variations in direction tuning, response strength, or any other tuning property that we examined. When microstimulation of a disparity-tuned site did affect direction judgments, the effects tended to be stronger at the preferred disparity of a stimulation site than at the nonpreferred disparity, indicating that monkeys can selectively monitor direction columns that are best tuned to an appropriate conjunction of parameters. We conclude that the contribution of neurons to behavior can depend strongly upon tuning to stimulus dimensions that appear to be irrelevant to the current task, and we suggest that these findings are best explained in terms of the strategy used by animals to perform the task.     

Perceptual “Read-Out” of Conjoined Direction and Disparity Maps in Extrastriate Area MT

Cortical neurons are frequently tuned to several stimulus dimensions, and many cortical areas contain intercalated maps of multiple variables. Relatively little is known about how information is “read out” of these multidimensional maps. For example, how does an organism extract information relevant to the task at hand from neurons that are also tuned to other, irrelevant stimulus dimensions? We addressed this question by employing microstimulation techniques to examine the contribution of disparity-tuned neurons in the middle temporal (MT) visual area to performance on a direction discrimination task. Most MT neurons are tuned to both binocular disparity and the direction of stimulus motion, and MT contains topographic maps of both parameters. We assessed the effect of microstimulation on direction judgments after first characterizing the disparity tuning of each stimulation site. Although the disparity of the stimulus was irrelevant to the required task, we found that microstimulation effects were strongly modulated by the disparity tuning of the stimulated neurons. For two of three monkeys, microstimulation of nondisparity-selective sites produced large biases in direction judgments, whereas stimulation of disparity-selective sites had little or no effect. The binocular disparity was optimized for each stimulation site, and our result could not be explained by variations in direction tuning, response strength, or any other tuning property that we examined. When microstimulation of a disparity-tuned site did affect direction judgments, the effects tended to be stronger at the preferred disparity of a stimulation site than at the nonpreferred disparity, indicating that monkeys can selectively monitor direction columns that are best tuned to an appropriate conjunction of parameters. We conclude that the contribution of neurons to behavior can depend strongly upon tuning to stimulus dimensions that appear to be irrelevant to the current task, and we suggest that these findings are best explained in terms of the strategy used by animals to perform the task.     

Stereoscopic mechanisms: binocular responses of the striate cells of cats to moving light and dark bars

New knowledge concerning the internal structure and response properties of the receptive fields of striate cells calls for a fresh appraisal of their binocular interactions in the interest of a better understanding of the neural mechanisms underlying binocular depth discrimination. Binocular position-disparity response profiles were recorded from 71 simple and B-cells in response to moving light and dark bars. Predominantly excitatory (PE) cells (N = 48) had disparity response profiles that were spatially closely similar to their respective monocular responses. In addition, the centrally located excitatory subregions were flanked on one or both sides by non-specific inhibitory regions. PE cells with a preferred stimulus orientation within 30 degrees of the vertical (N = 17) showed binocular facilitations with maximal values that were always more than twice (mean 3.3) the sum of the two monocular responses to the same stimuli and generally greater than the facilitations shown by cells with orientations more than 30 degrees from the vertical (N = 29; mean 2.2 times the sum of the respective monocular responses). The strength of the binocular facilitation depended on the stimulus contrast, the facilitation decreasing with increasing contrast. The receptive-field disparity distribution of the 31 PE cells capable of making significant horizontal disparity discriminations has standard deviations of 0.37 degrees and 0.40 degrees, respectively. Predominantly inhibitory cells (PI) (N = 23) showed two basic types of disparity response profile: symmetric (N = 17) and asymmetric (N = 6). Uncertainty regarding the precise location of the binocular fixation point in the anaesthetized and paralysed preparation made it difficult to categorize PI cells adequately.     

Luminance,Colour, Viewpoint and Border Enhanced Disparity Energy Model

The visual cortex is able to extract disparity information through the use of binocular cells. This process is reflected by the Disparity Energy Model, which describes the role and functioning of simple and complex binocular neuron populations, and how they are able to extract disparity. This model uses explicit cell parameters to mathematically determine preferred cell disparities, like spatial frequencies, orientations, binocular phases and receptive field positions. However, the brain cannot access such explicit cell parameters; it must rely on cell responses. In this article, we implemented a trained binocular neuronal population, which encodes disparity information implicitly. This allows the population to learn how to decode disparities, in a similar way to how our visual system could have developed this ability during evolution. At the same time, responses of monocular simple and complex cells can also encode line and edge information, which is useful for refining disparities at object borders. The brain should then be able, starting from a low-level disparity draft, to integrate all information, including colour and viewpoint perspective, in order to propagate better estimates to higher cortical areas.     

Stereoscopic Depth Perception Using a Model Based on the Primary Visual Cortex

This work describes an approach inspired by the primary visual cortex using the stimulus response of the receptive field profiles of binocular cells for disparity computation. Using the energy model based on the mechanism of log-Gabor filters for disparity encodings, we propose a suitable model to consistently represent the complex cells by computing the wide bandwidths of the cortical cells. This way, the model ensures the general neurophysiological findings in the visual cortex (V1), emphasizing the physical disparities and providing a simple selection method for the complex cell response. The results suggest that our proposed approach can achieve better results than a hybrid model with phase-shift and position-shift using position disparity alone.     

How behavioral constraints may determine optimal sensory representations

The sensory-triggered activity of a neuron is typically characterized in terms of a tuning curve, which describes the neuron's average response as a function of a parameter that characterizes a physical stimulus. What determines the shapes of tuning curves in a neuronal population? Previous theoretical studies and related experiments suggest that many response characteristics of sensory neurons are optimal for encoding stimulus-related information. This notion, however, does not explain the two general types of tuning profiles that are commonly observed: unimodal and monotonic. Here I quantify the efficacy of a set of tuning curves according to the possible downstream motor responses that can be constructed from them. Curves that are optimal in this sense may have monotonic or nonmonotonic profiles, where the proportion of monotonic curves and the optimal tuning-curve width depend on the general properties of the target downstream functions. This dependence explains intriguing features of visual cells that are sensitive to binocular disparity and of neurons tuned to echo delay in bats. The numerical results suggest that optimal sensory tuning curves are shaped not only by stimulus statistics and signal-to-noise properties but also according to their impact on downstream neural circuits and, ultimately, on behavior.     

A reaction-diffusion model to capture disparity selectivity in primary visual cortex

Decades of experimental studies are available on disparity selective cells in visual cortex of macaque and cat. Recently, local disparity map for iso-orientation sites for near-vertical edge preference is reported in area 18 of cat visual cortex. No experiment is yet reported on complete disparity map in V1. Disparity map for layer IV in V1 can provide insight into how disparity selective complex cell receptive field is organized from simple cell subunits. Though substantial amounts of experimental data on disparity selective cells is available, no model on receptive field development of such cells or disparity map development exists in literature. We model disparity selectivity in layer IV of cat V1 using a reaction-diffusion two-eye paradigm. In this model, the wiring between LGN and cortical layer IV is determined by resource an LGN cell has for supporting connections to cortical cells and competition for target space in layer IV. While competing for target space, the same type of LGN cells, irrespective of whether it belongs to left-eye-specific or right-eye-specific LGN layer, cooperate with each other while trying to push off the other type. Our model captures realistic 2D disparity selective simple cell receptive fields, their response properties and disparity map along with orientation and ocular dominance maps. There is lack of correlation between ocular dominance and disparity selectivity at the cell population level. At the map level, disparity selectivity topography is not random but weakly clustered for similar preferred disparities. This is similar to the experimental result reported for macaque. The details of weakly clustered disparity selectivity map in V1 indicate two types of complex cell receptive field organization.     

The development of the binocular depth cells in the secondary visual cortex of the lamb.

In most respects, the response properties of cells in the secondary visual cortex of the newborn lamb were indistinguishable from those in the adult. The cells were sharply selective to orientation; the orientation preferences were the same in each eye, and they varied systematically as the electrode penetrated the cortex. The receptive-field organization did not differ noticeably from that in adults, and complex, hypercomplex, and a few simple cells were all observed. The ocular dominance distribution was similar to that in the adult. Most importantly, binocular cells were found with disparate receptive fields even in newborn, visually inexperienced animals. As in the adult, the disparities were largely horizontal, and they appeared to be arranged in columns. Many of the cells responded preferentially to a binocular stimulus at a particular disparity setting (often approximately zero), but unlike those in the adult almost all the binocular cells in the newborn lamb would also respond monocularly, and the enhancement at the optimal disparity was less than in the adult. The full development of binocular selectivity took several weeks, and was blocked by binocular deprivation. We conclude that the basic wiring of stereoscopic mechanisms is innate, but the development of mature binocular interaction may depend on an adaptive process which makes use of the visual information received during binocular stimulation.     

Neural coding of 3D features of objects for hand action in the parietal cortex of the monkey.

In our previous studies of hand manipulation task-related neurons, we found many neurons of the parietal association cortex which responded to the sight of three-dimensional (3D) objects. Most of the task-related neurons in the AIP area (the lateral bank of the anterior intraparietal sulcus) were visually responsive and half of them responded to objects for manipulation. Most of these neurons were selective for the 3D features of the objects. More recently, we have found binocular visual neurons in the lateral bank of the caudal intraparietal sulcus (c-IPS area) that preferentially respond to a luminous bar or place at a particular orientation in space. We studied the responses of axis-orientation selective (AOS) neurons and surface-orientation selective (SOS) neurons in this area with stimuli presented on a 3D computer graphics display. The AOS neurons showed a stronger response to elongated stimuli and showed tuning to the orientation of the longitudinal axis. Many of them preferred a tilted stimulus in depth and appeared to be sensitive to orientation disparity and/or width disparity. The SOS neurons showed a stronger response to a flat than to an elongated stimulus and showed tuning to the 3D orientation of the surface. Their responses increased with the width or length of the stimulus. A considerable number of SOS neurons responded to a square in a random dot stereogram and were tuned to orientation in depth, suggesting their sensitivity to the gradient of disparity. We also found several SOS neurons that responded to a square with tilted or slanted contours, suggesting their sensitivity to orientation disparity and/or width disparity. Area c-IPS is likely to send visual signals of the 3D features of an object to area AIP for the visual guidance of hand actions.     

Spatial stereoresolution for depth corrugations may be set in primary visual cortex

Stereo "3D" depth perception requires the visual system to extract binocular disparities between the two eyes' images. Several current models of this process, based on the known physiology of primary visual cortex (V1), do this by computing a piecewise-frontoparallel local cross-correlation between the left and right eye's images. The size of the "window" within which detectors examine the local cross-correlation corresponds to the receptive field size of V1 neurons. This basic model has successfully captured many aspects of human depth perception. In particular, it accounts for the low human stereoresolution for sinusoidal depth corrugations, suggesting that the limit on stereoresolution may be set in primary visual cortex. An important feature of the model, reflecting a key property of V1 neurons, is that the initial disparity encoding is performed by detectors tuned to locally uniform patches of disparity. Such detectors respond better to square-wave depth corrugations, since these are locally flat, than to sinusoidal corrugations which are slanted almost everywhere. Consequently, for any given window size, current models predict better performance for square-wave disparity corrugations than for sine-wave corrugations at high amplitudes. We have recently shown that this prediction is not borne out: humans perform no better with square-wave than with sine-wave corrugations, even at high amplitudes. The failure of this prediction raised the question of whether stereoresolution may actually be set at later stages of cortical processing, perhaps involving neurons tuned to disparity slant or curvature. Here we extend the local cross-correlation model to include existing physiological and psychophysical evidence indicating that larger disparities are detected by neurons with larger receptive fields (a size/disparity correlation). We show that this simple modification succeeds in reconciling the model with human results, confirming that stereoresolution for disparity gratings may indeed be limited by the size of receptive fields in primary visual cortex.     

Direct extraction of curvature-based metric shape from stereo by view-modulated receptive fields

Any computation of metric surface structure from horizontal disparities depends on the viewing geometry, and analysing this dependence allows us to narrow down the choice of viable schemes. For example, all depth-based or slant-based schemes (i.e. nearly all existing models) are found to be unrealistically sensitive to natural errors in vergence. Curvature-based schemes avoid these problems and require only moderate, more robust view-dependent corrections to yield local object shape, without any depth coding. This fits the fact that humans are strikingly insensitive to global depth but accurate in discriminating surface curvature. The latter also excludes coding only affine structure. In view of new adaptation results, our goal becomes to directly extract retinotopic fields of metric surface curvatures (i.e. avoiding intermediate disparity curvature).To find a robust neural realisation, we combine new exact analysis with basic neural and psychophysical constraints. Systematic, step-by-step ‘design’ leads to neural operators which employ a novel family of ‘dynamic’ receptive fields (RFs), tuned to specific (bi-)local disparity structure. The required RF family is dictated by the non-Euclidean geometry that we identify as inherent in cyclopean vision. The dynamic RF-subfield patterns are controlled via gain modulation by binocular vergence and version, and parameterised by a cell-specific tuning to slant. Our full characterisation of the neural operators invites a range of new neurophysiological tests. Regarding shape perception, the model inverts widely accepted interpretations: It predicts the various types of errors that have often been mistaken for evidence against metric shape extraction.     

Binocularly driven neurons in the rostral part of the frog optic tectum

Summary Receptive field (RF) properties of binocular neurons lying in the rostral part of the optic tectum of the frog (Rana esculenta) were studied electrophysiologically using conventional visual stimuli. They were classified into five groups: group 1 neurons have indefinite RF; group 2 neurons are total-field (T6) neurons; group 3 neurons have RFs covering a quadrant of the frontal visual field; group 4 neurons resemble T 1(1) and T 1(3) subclasses described earlier; and finally group 5 neurons look like small-field binocular neurons and are called T7(B). Moreover, RF disparity measurements conducted in all groups suggest that group 4 neurons support the estimation of binocular distance. This problem is discussed.Abbreviation RF receptive field     

Striate, extrastriate and collicular processing of spatial disparity cues

The spatial disparity sensitivity of single units in the primary visual cortex (17-18 border), in extrastriate area 19 and in the superficial layers of the superior colliculus of the cat brain were compared in the present study. Unit recordings were performed in paralyzed and anesthetized animals. Centrally located receptive fields were mapped, separated using prisms and then stimulated simultaneously using two luminous bars optimally adjusted to the size of the excitatory receptive fields. In the three regions studied, cells selective to spatial disparity were found and four classes of disparity sensitivity profiles emerged. Although the disparity sensitivity profiles of the cells in the three regions appeared to have the same general shape, selectivity was clearly different. Cells at the 17-18 border were sharply tuned, those of area 19 were not only less numerous but also less well tuned and collicular cells exhibited coarse selectivity. These differences in selectivity appear to be linked to the projection pattern of the X, Y and W systems to these regions and the roles that these cells might play in vision.     

Brain activation difference evoked by different binocular disparities of stereograms: An fMRI study

The binocular disparity of two retina images is a main cue of stereoscopic vision. However, the global dependency between brain response and binocular disparity still remains unclear. Here, we used functional Magnetic Resonance Imaging (fMRI) to identify stereopsis-related brain regions with a modified Random Dot Stereogram (RDS) and plotted the activation variation curves under different disparity size. In order to eliminate the confounding shape difference between the stereogram and the plane, commonly seen in RDS, we modified the RDS to a checkerboard version. We found that V3A, V7 and MT+/V5 in dorsal visual stream were activated in stereoscopic experiment, while little activation was found in ventral visual regions. According to the activation trends, 13 subjects were divided into three groups: 5 subjects with turning points (a shift from increased to decreased activation), 5 subjects without turning points and 3 subjects with activation unrelated to disparity. We inferred that the dorsal visual stream primarily processes spatial depth information, rather than shape information.     

视差检测：简单细胞、复杂细胞及能量模型

立体视觉信息的处理在于皮层双眼性细胞的活动.皮层中简单细胞对视差的编码方式被认为有两种：位置差（position shift）和相位差（phase shift）,但简单细胞并不适合作为视差检测器.对一些复杂细胞的视差响应特性的生理研究,发现复杂细胞是一种比较适合的视差检测器.模型的研究提出基于这类简单细胞的复杂细胞能量模型,可以很好的检测视差,并可以较好的解释一些生理现象.     

Effects of Prism Eyeglasses on Objective and Subjective Fixation Disparity

In optometry of binocular vision, the question may arise whether prisms should be included in eyeglasses to compensate an oculomotor and/or sensory imbalance between the two eyes. The corresponding measures of objective and subjective fixation disparity may be reduced by the prisms, or the adaptability of the binocular vergence system may diminish effects of the prisms over time. This study investigates effects of wearing prisms constantly for about 5 weeks in daily life. Two groups of 12 participants received eyeglasses with prisms having either a base-in direction or a base-out direction with an amount up to 8 prism diopters. Prisms were prescribed based on clinical fixation disparity test plates at 6 m. Two dependent variables were used: (1) subjective fixation disparity was indicated by a perceived offset of dichoptic nonius lines that were superimposed on the fusion stimuli and (2) objective fixation disparity was measured with a video based eye tracker relative to monocular calibration. Stimuli were presented at 6 m and included either central or more peripheral fusion stimuli. Repeated measurements were made without the prisms and with the prisms after about 5 weeks of wearing these prisms. Objective and subjective fixation disparity were correlated, but the type of fusion stimulus and the direction of the required prism may play a role. The prisms did not reduce the fixation disparity to zero, but induced significant changes in fixation disparity with large effect sizes. Participants receiving base-out prisms showed hypothesized effects, which were concurrent in both types of fixation disparity. In participants receiving base-in prisms, the individual effects of subjective and objective effects were negatively correlated: the larger the subjective (sensory) effect, the smaller the objective (motor) effect. This response pattern was related to the vergence adaptability, i.e. the individual fusional vergence reserves.     

A model of neural network extracting binocular parallax

A model of neural network extracting binocular parallax is proposed. It is a multilayered network composed of analog threshold elements. Three types of binocular neurons are included in this model. They are binocular simple neurons, binocular gate neurons and binocular depth neurons. The final layers of this model consist of elements which correspond to the binocular depth neurons. The performance of the model has been simulated on a digital computer. The results of the computer simulation show that every element of this model acts like neurons found in cat's and monkey's visual system and this model extracts binocular parallax caused by simple line components satisfactorily.