Visual depth encoding in populations of neurons with localized receptive fields

 Stereopsis is the ability to perceive three-dimensional structure from disparities between the two-dimensional retinal images. Although disparity-sensitive neurons have been proposed as a neural representation of this ability many years ago, it is still difficult to link all qualities of stereopsis to properties of the neural correlate of binocular disparities. The present study wants to support efforts directed at closing the gap between electrophysiology and psychophysics. Populations of disparity-sensitive neurons in V1 were simulated using the energy-neuron model. Responses to different types of stimuli were evaluated with an efficient statistical estimator and related to psychophysical findings. The representation of disparity in simulated population responses appeared to be very robust. Small populations allowed good depth discrimination. Two types of energy neurons (phase- and position-type models) that are discussed as possible neural implementations of disparity-selectivity could be compared to each other. Phase-type coding was more robust and could explain a tendency towards zero disparity in degenerated stimuli and, for high-pass stimuli, exhibited the breakdown of disparity discrimination at a maximum disparity value. Contrast-inverted stereograms led to high variances in disparity representation, which is a possible explanation of the absence of depth percepts in large contrast-inverted stimuli. Our study suggests that nonlocal interactions destroy depth percepts in large contrast-inverted stereograms, although these percepts occur for smaller stimuli of the same class. Received: 21 December 2001 / Accepted: 29 April 2002 RID="*" ID="*" Present address: Bayer AG BTS-PT-MVT-MKM, Geb. K9, 51368 Leverkusen, Germany Acknowledgement. This work was supported by a scholarship from the Studienstiftung des deutschen Volkes to J.L. Correspondence to: J. Lippert (e-mail: joerg.lippert.jl@bayer-ag.de)     

Depth Perception Not Found in Human Observers for Static or Dynamic Anti-Correlated Random Dot Stereograms

One of the greatest challenges in visual neuroscience is that of linking neural activity with perceptual experience. In the case of binocular depth perception, important insights have been achieved through comparing neural responses and the perception of depth, for carefully selected stimuli. One of the most important types of stimulus that has been used here is the anti-correlated random dot stereogram (ACRDS). In these stimuli, the contrast polarity of one half of a stereoscopic image is reversed. While neurons in cortical area V1 respond reliably to the binocular disparities in ACRDS, they do not create a sensation of depth. This discrepancy has been used to argue that depth perception must rely on neural activity elsewhere in the brain. Currently, the psychophysical results on which this argument rests are not clear-cut. While it is generally assumed that ACRDS do not support the perception of depth, some studies have reported that some people, some of the time, perceive depth in some types of these stimuli. Given the importance of these results for understanding the neural correlates of stereopsis, we studied depth perception in ACRDS using a large number of observers, in order to provide an unambiguous conclusion about the extent to which these stimuli support the perception of depth. We presented observers with random dot stereograms in which correlated dots were presented in a surrounding annulus and correlated or anti-correlated dots were presented in a central circular region. While observers could reliably report the depth of the central region for correlated stimuli, we found no evidence for depth perception in static or dynamic anti-correlated stimuli. Confidence ratings for stereoscopic perception were uniformly low for anti-correlated stimuli, but showed normal variation with disparity for correlated stimuli. These results establish that the inability of observers to perceive depth in ACRDS is a robust phenomenon.     

A psychophysical and computational analysis of intensity–based stereo

We describe two psychophysical experiments testing predictions of the square difference mechanism we have previously proposed for intensity–based stereo. Experiment 1 assesses the relative contributions of disparity and contrast to intensity–based stereo by measuring detection thresholds. The product of disparity and contrast at threshold is shown to be constant. In experiment 2, we measure quantitatively the global depth position perceived in stereograms of curved, smoothly shaded surfaces. The results show that disparity averaging over the surface involves a contrast-dependent weighting function. The results from both experiments are consistent with predictions derived from the square difference mechanism. The relation of this mechanism to feature correspondence stereopsis and shape–from–shading is discussed and a general framework for assessing the modularity of stereopsis is presented. Received: 9 June 1995 / Accepted in revised form: 3 June 1996     

The direction of retinal motion facilitates binocular stereopsis.

Visual information from binocular disparity and from relative motion provide information about three-dimensional structure and layout of the world. Although the mechanisms that process these cues have typically been studied independently, there is now a substantial body of evidence that suggests that they interact in the visual pathway. This paper investigates one advantage of such an interaction: whether retinal motion can be used as a matching constraint in the binocular correspondence process. Stimuli that contained identical disparity and motion signals but which differed in their fine-scale correlation were created to establish whether the direction, or the speed, of motion could enhance performance in a psychophysical task in which binocular matching is a limiting factor. The results of these experiments provide clear evidence that different directions of motion, but not different speeds, are processed separately in stereopsis. The results fit well with properties of neurons early in the cortical visual pathway which are thought to be involved in determining local matches between features in the two eyes'' images.     

Size-disparity correlation in human binocular depth perception.

To use the small horizontal disparities between images projected to the eyes for the recovery of three-dimensional information, our visual system must first identify which feature in one eye's image corresponds with which in the other. The earliest level of disparity processing in primates (V1) contains cells that are spatial-frequency tuned. If such cells have a disparity range that covers only a single period of their mean tuning frequency, there will always be exactly one potential match within this range. Here, this 'size-disparity' hypothesis was tested by measuring the contrast sensitivity of stereopsis as a function of disparity for single bandpass-filtered items. It was found that thresholds were low and relatively constant up to disparities an order of magnitude larger than is predicted by this constraint. Furthermore, peak sensitivity was relatively independent of spatial frequency. A control experiment showed that binocular correlation of the carrier is necessary for this task. In a third experiment, the maximum disparity that supports threshold performance was compared for an isolated bandpass item and bandpass-filtered noise. This limit was found to be five times larger for the isolated stimuli. In summary, these findings show that the initial stage of disparity detection is not limited by the size-disparity constraint. For stimuli with multiple false targets, however, processes subsequent to this stage reduce the disparity range over which the correspondence problem can be solved.     

A computational model of stereoscopic prey capture in praying mantises

We present a simple model which can account for the stereoscopic sensitivity of praying mantis predatory strikes. The model consists of a single “disparity sensor”: a binocular neuron sensitive to stereoscopic disparity and thus to distance from the animal. The model is based closely on the known behavioural and neurophysiological properties of mantis stereopsis. The monocular inputs to the neuron reflect temporal change and are insensitive to contrast sign, making the sensor insensitive to interocular correlation. The monocular receptive fields have a excitatory centre and inhibitory surround, making them tuned to size. The disparity sensor combines inputs from the two eyes linearly, applies a threshold and then an exponent output nonlinearity. The activity of the sensor represents the model mantis’s instantaneous probability of striking. We integrate this over the stimulus duration to obtain the expected number of strikes in response to moving targets with different stereoscopic disparity, size and vertical disparity. We optimised the parameters of the model so as to bring its predictions into agreement with our empirical data on mean strike rate as a function of stimulus size and disparity. The model proves capable of reproducing the relatively broad tuning to size and narrow tuning to stereoscopic disparity seen in mantis striking behaviour. Although the model has only a single centre-surround receptive field in each eye, it displays qualitatively the same interaction between size and disparity as we observed in real mantids: the preferred size increases as simulated prey distance increases beyond the preferred distance. We show that this occurs because of a stereoscopic “false match” between the leading edge of the stimulus in one eye and its trailing edge in the other; further work will be required to find whether such false matches occur in real mantises. Importantly, the model also displays realistic responses to stimuli with vertical disparity and to pairs of identical stimuli offering a “ghost match”, despite not being fitted to these data. This is the first image-computable model of insect stereopsis, and reproduces key features of both neurophysiology and striking behaviour.     

Influence of Correspondence Noise and Spatial Scaling on the Upper Limit for Spatial Displacement in Fully-Coherent Random-Dot Kinematogram Stimuli

Correspondence noise is a major factor limiting direction discrimination performance in random-dot kinematograms [1]. In the current study we investigated the influence of correspondence noise on Dmax, which is the upper limit for the spatial displacement of the dots for which coherent motion is still perceived. Human direction discrimination performance was measured, using 2-frame kinematograms having leftward/rightward motion, over a 200-fold range of dot-densities and a four-fold range of dot displacements. From this data Dmax was estimated for the different dot densities tested. A model was proposed to evaluate the correspondence noise in the stimulus. This model summed the outputs of a set of elementary Reichardt-type local detectors that had receptive fields tiling the stimulus and were tuned to the two directions of motion in the stimulus. A key assumption of the model was that the local detectors would have the radius of their catchment areas scaled with the displacement that they were tuned to detect; the scaling factor k linking the radius to the displacement was the only free parameter in the model and a single value of k was used to fit all of the psychophysical data collected. This minimal, correspondence-noise based model was able to account for 91% of the variability in the human performance across all of the conditions tested. The results highlight the importance of correspondence noise in constraining the largest displacement that can be detected.     

A laminar cortical model of stereopsis and 3D surface perception: closure and da Vinci stereopsis

A laminar cortical model of stereopsis and 3D surface perception is developed and simulated. The model describes how monocular and binocular oriented filtering interact with later stages of 3D boundary formation and surface filling-in in the LGN and cortical areas V1, V2, and V4. It proposes how interactions between layers 4, 3B, and 2/3 in V1 and V2 contribute to stereopsis, and how binocular and monocular information combine to form 3D boundary and surface representations. The model includes two main new developments: (1) It clarifies how surface-to-boundary feedback from V2 thin stripes to pale stripes helps to explain data about stereopsis. This feedback has previously been used to explain data about 3D figure-ground perception. (2) It proposes that the binocular false match problem is subsumed under the Gestalt grouping problem. In particular, the disparity filter, which helps to solve the correspondence problem by eliminating false matches, is realized using inhibitory interneurons as part of the perceptual grouping process by horizontal connections in layer 2/3 of cortical area V2. The enhanced model explains all the psychophysical data previously simulated by Grossberg and Howe (2003), such as contrast variations of dichoptic masking and the correspondence problem, the effect of interocular contrast differences on stereoacuity, Panum's limiting case, the Venetian blind illusion, stereopsis with polarity-reversed stereograms, and da Vinci stereopsis. It also explains psychophysical data about perceptual closure and variations of da Vinci stereopsis that previous models cannot yet explain.     

Electrophysiological Correlates of Binocular Stereo Depth without Binocular Disparities

A small region of background presented to only one eye in an otherwise binocular display may, under certain conditions, be resolved in the visual system by interpreting the region as a small gap between two similar objects placed at different depths, with the gap hidden in one eye by parallax. This has been called monocular gap stereopsis. We investigated the electrophysiological correlate of this type of stereopsis by means of sum potential recordings in 12 observers, comparing VEP's for this stimulus ("Gillam Stereo", Author BG has strong reservations about this term) with those for similar stimuli containing disparity based depth and with no depth (flat). In addition we included several control stimuli. The results show a pronounced early negative potential at a latency of around 170 ms (N170) for all stimuli containing non- identical elements, be they a difference caused by binocular disparity or by completely unmatched monocular contours. A second negative potential with latency around 270 ms (N270), on the other hand, is present only with stimuli leading to fusion and the perception of depth. This second component is similar for disparity-based stereopsis and monocular gap, or "Gillam Stereo" although slightly more pronounced for the former. We conjecture that the first component is related to the detection of differences between the images of the two eyes that may then either be fused, leading to stereopsis and the corresponding second potential, or else to inhibition and rivalry without a later trace in the VEP. The finding that that "Gillam Stereo" leads to cortical responses at the same short latencies as disparity based stereopsis indicates that it may partly rely on quite early cortical mechanisms.     

A transduction model of the Meissner corpuscle

I propose a transduction model of the Meissner corpuscle that integrates ideas put forth by Freeman and Johnson and results obtained by Looft. The principal development in the present model is its specification that RA receptor potentials are updated as a linear function of stimulus velocity above baseline; the model thus readily accommodates non-sinusoidal input. It also incorporates modifications to Freeman and Johnson's model proposed by Slavík and Bell, namely a period of refractoriness lasting 1 ms followed by a period of hyperexcitability lasting 13.5 ms. The model is applied to various psychophysical and physiological situations: psychophysical threshold vs. frequency, RA afferent impulse rates vs. intensity, impulse regularity vs. frequency, phase retardation vs. frequency, and responses to non-repeating noise and to complex stimuli. Model output closely matches psychophysical and neurophysiological data. The proposed model thus reliably predicts RA afferent responses to arbitrary stimuli and may facilitate the development of theories relating psychophysical phenomena to their underlying neural representations.     

Disparity tuning as simulated by a neural net

Previous research has suggested that the processing of binocular disparity in complex cells may be described with an energy formalism. The energy formalism allows for a representation of disparity by differences in the position or in the phase of monocular receptive subfields of binocular cells, or by combination of these two types. We studied the coding of disparities with an approach complementary to previous algorithmic investigations. Since realization of these representations is probably not genetically determined but learned during ontogeny, we used backpropagation networks to study which of these three possibilities were realized within neural nets. Three types of networks were trained with noise patterns in analogy to the three types of energy models. The networks learned the task and generalized to untrained correlated noise pattern input. Outputs were broadly tuned to spatial frequency and did not respond to anti-correlated noise patterns. Although the energy model was not explicitly implemented, we could analyze the outputs of the networks using predictions of the energy formalism. After learning was completed, the model neurons preferred position shifts over phase shifts in representing disparity. We discuss the general meaning of these findings and the correspondences and deviations between the energy model, V1 neurons, and our networks. Received: 6 August 1999 / Accepted in revised form: 26 January 2000     

Adaptive computational models of fast learning of motion direction discrimination

In a previous study, we found that subjects' performance in a task of direction discrimination in stochastic motion stimuli shows fast improvement in the absence of feedback and the learned ability is retained over a period of time. We model this learning using two unsupervised approaches: a clustering model that learns to accommodate the motion noise, and an averaging model that learns to ignore the noise. Extensive simulations with the models show performance similar to psychophysical results.     

Taste- and odor-reactivity in elderly demented patients

Previous studies demonstrated that hedonically different chemical(taste or smell) stimuli induceinnate, inherited, differentialand distinct fixed reflectory motion features in the oral andfacial area. In the present study 20 elderly demented patients,suffering from ‘probable’ or ‘possible’Alzheimer's disease, and 20 normally functioning elderly subjectswere tested. The facial expressive behavioral reactions triggeredby a set of common gustatory and olfactory stimuli were videotaped.Both psychophysical and stimulus-dependent behavioral responseswere obtained from the control group, while for the dementedpatients only behavioral reactions were recorded. Results revealedthat: (i) severely demented elderly subjects displayed differentialand distinct orofacial responses indicating ‘acceptance’and ‘aversion’. These were found to be analogousto but less intense than those displayed by control age mates;(ii) the duration of responses induced by aversive tastes islonger than that triggered by pleasant or indifferent ones,for both groups; (iii) all gustatory and olfactory stimuli triggera longer lasting behavioral response in demented than in normalsubjects; and (iv) psychophysical and behavioral responses ofthe control subjects gave similar results for taste- and odor-hedonicsas well as for their intensity. This finding clearly indicatesthe validity of the alternative use of psychophysical and behavioraltesting procedures.     

体视感觉“崩溃”的阀值

视差是产生体视感觉的主要因素.但视差过大时这种体视感觉也不能存在.这时随着双眼视差的增大从融合为单一像到成复像,而引起体视“崩溃”.本文对正常人和体视欠缺者用心理物理试验方法结合视觉诱发电位(VEP)分析,测量了人眼体视“崩溃”的视差上限阈值.并在改变刺激图形亮度和面积时加以比较.我们的结果表明正常人体视上限阈值,其视差高于2度,在某些情况下这个“崩溃”阈值达到3度.体视欠缺者的“崩溃”阈值约为1.5度.亮度和面积变小会使体视“崩溃”阈值下降,但在这种情况下体视欠缺者的“崩溃”阈值不会下降.可以认为1.5度是人的最低上限阈值·体视存在时VEP的N峰潜伏期约为220—300ms,P_3峰在280—340ms之间.无体视现象(视差为零和过大)这两个峰的潜伏期明显加大.     

Noise improves transfer of near-threshold, phase-locked activity of the cochlear nerve: evidence for stochastic resonance?

 Stochastic resonance can be described as improved detection of weak periodic stimuli by a dynamic nonlinear system, resulting from the simultaneous presentation of a restricted dynamic range of low-intensity noise. This property has been reported in simple physical and biological activities. The present study describes data consistent with the interpretation that stochastic resonance can be observed in the response of cochlear neurons. These experiments utilized low levels (−5 to 25 dB SPL) of stimuli and noise (5 to 30 dB SPL). Stimuli consisted of simultaneously presented 8 kHz (F ₁) and 8.8 kHz (F ₂) tone bursts, which generated an 800 Hz F ₂–F ₁ cochlear nerve envelope ensemble response in the gerbil. The mean response threshold was approximately −3 dB SPL. Simultaneous presentation of a low-intensity wideband noise increased the amplitude of this response. This was observed with tonal stimuli having intensities of 0–5 dB SPL; responses to stimulus levels >10 dB were attenuated by noise. Response amplitude was increased by noise levels of 10–15 dB; the amplitude was unaffected by lower levels of noise, and decreased in the presence of higher noise levels. These properties are compatible with those of stochastic resonance. Accepted: 11 March 1999     

Stereopsis activates V3A and caudal intraparietal areas in macaques and humans

Stereopsis, the perception of depth from small differences between the images in the two eyes, provides a rich model for investigating the cortical construction of surfaces and space. Although disparity-tuned cells have been found in a large number of areas in macaque visual cortex, stereoscopic processing in these areas has never been systematically compared using the same stimuli and analysis methods. In order to examine the global architecture of stereoscopic processing in primate visual cortex, we studied fMRI activity in alert, fixating human and macaque subjects. In macaques, we found strongest activation to near/far compared to zero disparity in areas V3, V3A, and CIPS. In humans, we found strongest activation to the same stimuli in areas V3A, V7, the V4d topolog (V4d-topo), and a caudal parietal disparity region (CPDR). Thus, in both primate species a small cluster of areas at the parieto-occipital junction appears to be specialized for stereopsis.     

A Dynamic Neural Field Model of Mesoscopic Cortical Activity Captured with Voltage-Sensitive Dye Imaging

A neural field model is presented that captures the essential non-linear characteristics of activity dynamics across several millimeters of visual cortex in response to local flashed and moving stimuli. We account for physiological data obtained by voltage-sensitive dye (VSD) imaging which reports mesoscopic population activity at high spatio-temporal resolution. Stimulation included a single flashed square, a single flashed bar, the line-motion paradigm – for which psychophysical studies showed that flashing a square briefly before a bar produces sensation of illusory motion within the bar – and moving squares controls. We consider a two-layer neural field (NF) model describing an excitatory and an inhibitory layer of neurons as a coupled system of non-linear integro-differential equations. Under the assumption that the aggregated activity of both layers is reflected by VSD imaging, our phenomenological model quantitatively accounts for the observed spatio-temporal activity patterns. Moreover, the model generalizes to novel similar stimuli as it matches activity evoked by moving squares of different speeds. Our results indicate that feedback from higher brain areas is not required to produce motion patterns in the case of the illusory line-motion paradigm. Physiological interpretation of the model suggests that a considerable fraction of the VSD signal may be due to inhibitory activity, supporting the notion that balanced intra-layer cortical interactions between inhibitory and excitatory populations play a major role in shaping dynamic stimulus representations in the early visual cortex.     

Two-dimensional substructure of stereo and motion interactions in macaque visual cortex

The analysis of object motion and stereoscopic depth are important tasks that are begun at early stages of the primate visual system. Using sparse white noise, we mapped the receptive field substructure of motion and disparity interactions in neurons in V1 and MT of alert monkeys. Interactions in both regions revealed subunits similar in structure to V1 simple cells. For both motion and stereo, the scale and shape of the receptive field substructure could be predicted from conventional tuning for bars or dot-field stimuli, indicating that the small-scale interactions were repeated across the receptive fields. We also found neurons in V1 and in MT that were tuned to combinations of spatial and temporal binocular disparities, suggesting a possible neural substrate for the perceptual Pulfrich phenomenon. Our observations constrain computational and developmental models of motion-stereo integration.     

Deep tectal cells in pigeons respond to kinematograms

Summary Deep tectal neurons in pigeons respond selectively to moving visual stimuli, and are inhibited by large background patterns moved in-phase with these stimuli. In this investigation we demonstrate that these same deep tectal neurons respond equally well to kinematograms as they do to traditional luminance contrast stimuli typically employed in visual experiments.Computer generated kinematograms, the motion domain equivalents of random dot stereograms, were used as stimuli in these experiments. These kinematograms, where a small centrally located set of random dots is moved coherently in one direction while the remaining dots are moved in a different direction, thus constitute a pure motion stimulus where the stimulus form is only visible in the dynamic pattern, but does not exist on any single frame. Both object configured and hole configured kinematograms were employed; the former appearing as regions of texture moving over, or in front of, the background texture, while the latter appear as windows through which a more distant textured surface is revealed.Extracellular recordings from isolated deep tectal cells showed that all units responded in a very similar manner whether the stimulus was an object configured kinematogram or the more traditional luminance contrast variety. This similarity included directional selectivity, the in-phase inhibition anti-phase facilitation effect, and sensitivity to opposed motion independent of direction. However, when the kinematograms were configured as holes none of the units tested responded to these stimuli. The significance of these observations for tectal functioning, image segmentation through motion and animal camouflage is discussed.