A Bayesian model of stereopsis depth and motion direction discrimination

 The extraction of stereoscopic depth from retinal disparity, and motion direction from two-frame kinematograms, requires the solution of a correspondence problem. In previous psychophysical work [Read and Eagle (2000) Vision Res 40: 3345–3358], we compared the performance of the human stereopsis and motion systems with correlated and anti-correlated stimuli. We found that, although the two systems performed similarly for narrow-band stimuli, broad-band anti-correlated kinematograms produced a strong perception of reversed motion, whereas the stereograms appeared merely rivalrous. I now model these psychophysical data with a computational model of the correspondence problem based on the known properties of visual cortical cells. Noisy retinal images are filtered through a set of Fourier channels tuned to different spatial frequencies and orientations. Within each channel, a Bayesian analysis incorporating a prior preference for small disparities is used to assess the probability of each possible match. Finally, information from the different channels is combined to arrive at a judgement of stimulus disparity. Each model system – stereopsis and motion – has two free parameters: the amount of noise they are subject to, and the strength of their preference for small disparities. By adjusting these parameters independently for each system, qualitative matches are produced to psychophysical data, for both correlated and anti-correlated stimuli, across a range of spatial frequency and orientation bandwidths. The motion model is found to require much higher noise levels and a weaker preference for small disparities. This makes the motion model more tolerant of poor-quality reverse-direction false matches encountered with anti-correlated stimuli, matching the strong perception of reversed motion that humans experience with these stimuli. In contrast, the lower noise level and tighter prior preference used with the stereopsis model means that it performs close to chance with anti-correlated stimuli, in accordance with human psychophysics. Thus, the key features of the experimental data can be reproduced assuming that the motion system experiences more effective noise than the stereoscopy system and imposes a less stringent preference for small disparities. Received: 2 March 2001 / Accepted in revised form: 5 July 2001     

Disambiguating Multi–Modal Scene Representations Using Perceptual Grouping Constraints

In its early stages, the visual system suffers from a lot of ambiguity and noise that severely limits the performance of early vision algorithms. This article presents feedback mechanisms between early visual processes, such as perceptual grouping, stereopsis and depth reconstruction, that allow the system to reduce this ambiguity and improve early representation of visual information. In the first part, the article proposes a local perceptual grouping algorithm that — in addition to commonly used geometric information — makes use of a novel multi–modal measure between local edge/line features. The grouping information is then used to: 1) disambiguate stereopsis by enforcing that stereo matches preserve groups; and 2) correct the reconstruction error due to the image pixel sampling using a linear interpolation over the groups. The integration of mutual feedback between early vision processes is shown to reduce considerably ambiguity and noise without the need for global constraints.     

A laminar cortical model of stereopsis and 3D surface perception: closure and da Vinci stereopsis

A laminar cortical model of stereopsis and 3D surface perception is developed and simulated. The model describes how monocular and binocular oriented filtering interact with later stages of 3D boundary formation and surface filling-in in the LGN and cortical areas V1, V2, and V4. It proposes how interactions between layers 4, 3B, and 2/3 in V1 and V2 contribute to stereopsis, and how binocular and monocular information combine to form 3D boundary and surface representations. The model includes two main new developments: (1) It clarifies how surface-to-boundary feedback from V2 thin stripes to pale stripes helps to explain data about stereopsis. This feedback has previously been used to explain data about 3D figure-ground perception. (2) It proposes that the binocular false match problem is subsumed under the Gestalt grouping problem. In particular, the disparity filter, which helps to solve the correspondence problem by eliminating false matches, is realized using inhibitory interneurons as part of the perceptual grouping process by horizontal connections in layer 2/3 of cortical area V2. The enhanced model explains all the psychophysical data previously simulated by Grossberg and Howe (2003), such as contrast variations of dichoptic masking and the correspondence problem, the effect of interocular contrast differences on stereoacuity, Panum's limiting case, the Venetian blind illusion, stereopsis with polarity-reversed stereograms, and da Vinci stereopsis. It also explains psychophysical data about perceptual closure and variations of da Vinci stereopsis that previous models cannot yet explain.     

The influence of the retinal inhomogeneity on the perception of spatial patterns

From the fact that the retina is rather inhomogeneous, it can be inferred that the perception of spatial patterns of appreciable extent will be dependent on the retinal location. Anatomical, electrophysiological and psychophysical findings substantiate the claim that the retina is very inhomogeneous of composition. In order to investigate the influence of this inhomogeneity on the perception of patterns, a model of spatiotemporal signal processing in the retina was developed on the basis of a paradigm for the Weber type adaptation. Such “scaling-ensembles” proved successful in the prediction of spatiotemporal modulation transfer in the human fovea (Koenderink et al., 1971). One prediction of the present model is that certain spatial patterns are optimally detected at well defined retinal locations, dependent on the spatial frequency content of the stimulus. A confrontation of the model's predictions with measurements published by Bryngdahl (1966) enabled us to estimate some of the relevant parameters of the retinal receptive fields as a function of the eccentricity. We obtained estimates that compare reasonably well with previously known values; for instance with values of acuity and anatomical measurements. The present discussion bears relevance on the question of whether the retina is composed of independently tuned spatial frequency filters at any retinal location, or whether the tuning is with respect to the eccentricity.     

Multi-scale keypoints in V1 and beyond: object segregation, scale selection, saliency maps and face detection

End-stopped cells in cortical area V1, which combine outputs of complex cells tuned to different orientations, serve to detect line and edge crossings, singularities and points with large curvature. These cells can be used to construct retinotopic keypoint maps at different spatial scales (level-of-detail). The importance of the multi-scale keypoint representation is studied in this paper. It is shown that this representation provides very important information for object recognition and face detection. Different grouping operators can be used for object segregation and automatic scale selection. Saliency maps for focus-of-attention can be constructed. Such maps can be employed for face detection by grouping facial landmarks at eyes, nose and mouth. Although a face detector can be based on processing within area V1, it is argued that such an operator must be embedded into dorsal and ventral data streams, to and from higher cortical areas, for obtaining translation-, rotation- and scale-invariant detection.     

不同空间频率通道的深度知觉及其质地的影响

我们设计了四种不同质地的随机点立体图对(RDS),并按照Wilson的空间频率通道的频带,分别利用Butterworth滤波器及Gabor函数对之进行了数字滤波处理,然后进行各种左右眼配对观察.得结果如下:1、Gabor函数滤波后图对的匹配结果好于Butterworth滤波器滤波后的图对.2、滤波后图对的匹配效果与图象质地有关.3、不同滤波的图对匹配大多也能形成立体感.4、在体视不同通道图对的匹配中,右眼对图象的高频成分更为敏感.     

A computational model of stereoscopic prey capture in praying mantises

We present a simple model which can account for the stereoscopic sensitivity of praying mantis predatory strikes. The model consists of a single “disparity sensor”: a binocular neuron sensitive to stereoscopic disparity and thus to distance from the animal. The model is based closely on the known behavioural and neurophysiological properties of mantis stereopsis. The monocular inputs to the neuron reflect temporal change and are insensitive to contrast sign, making the sensor insensitive to interocular correlation. The monocular receptive fields have a excitatory centre and inhibitory surround, making them tuned to size. The disparity sensor combines inputs from the two eyes linearly, applies a threshold and then an exponent output nonlinearity. The activity of the sensor represents the model mantis’s instantaneous probability of striking. We integrate this over the stimulus duration to obtain the expected number of strikes in response to moving targets with different stereoscopic disparity, size and vertical disparity. We optimised the parameters of the model so as to bring its predictions into agreement with our empirical data on mean strike rate as a function of stimulus size and disparity. The model proves capable of reproducing the relatively broad tuning to size and narrow tuning to stereoscopic disparity seen in mantis striking behaviour. Although the model has only a single centre-surround receptive field in each eye, it displays qualitatively the same interaction between size and disparity as we observed in real mantids: the preferred size increases as simulated prey distance increases beyond the preferred distance. We show that this occurs because of a stereoscopic “false match” between the leading edge of the stimulus in one eye and its trailing edge in the other; further work will be required to find whether such false matches occur in real mantises. Importantly, the model also displays realistic responses to stimuli with vertical disparity and to pairs of identical stimuli offering a “ghost match”, despite not being fitted to these data. This is the first image-computable model of insect stereopsis, and reproduces key features of both neurophysiology and striking behaviour.     

Analysis of spiking synchrony in visual cortex reveals distinct types of top-down modulation signals for spatial and object-based attention

The activity of a border ownership selective (BOS) neuron indicates where a foreground object is located relative to its (classical) receptive field (RF). A population of BOS neurons thus provides an important component of perceptual grouping, the organization of the visual scene into objects. In previous theoretical work, it has been suggested that this grouping mechanism is implemented by a population of dedicated grouping (“G”) cells that integrate the activity of the distributed feature cells representing an object and, by feedback, modulate the same cells, thus making them border ownership selective. The feedback modulation by G cells is thought to also provide the mechanism for object-based attention. A recent modeling study showed that modulatory common feedback, implemented by synapses with N-methyl-D-aspartate (NMDA)-type glutamate receptors, accounts for the experimentally observed synchrony in spike trains of BOS neurons and the shape of cross-correlations between them, including its dependence on the attentional state. However, that study was limited to pairs of BOS neurons with consistent border ownership preferences, defined as two neurons tuned to respond to the same visual object, in which attention decreases synchrony. But attention has also been shown to increase synchrony in neurons with inconsistent border ownership selectivity. Here we extend the computational model from the previous study to fully understand these effects of attention. We postulate the existence of a second type of G-cell that represents spatial attention by modulating the activity of all BOS cells in a spatially defined area. Simulations of this model show that a combination of spatial and object-based mechanisms fully accounts for the observed pattern of synchrony between BOS neurons. Our results suggest that modulatory feedback from G-cells may underlie both spatial and object-based attention.     

Energy filters,motion uncertainty,and motion sensitive cells in the visual cortex: a mathematical analysis

Energy filters are tuned to space-time frequency orientations. In order to compute velocity it is necessary to use a collection of filters, each tuned to a different space-time frequency. Here we analyze, in a probabilistic framework, the properties of the motion uncertainty. Its lower bound, which can be explicitly computed through the Cramér-Rao inequality, will have different values depending on the filter parameters. We show for the Gabor filter that, in order to minimize the motion uncertainty, the spatial and temporal filter sizes cannot be arbitrarily chosen; they are only allowed to vary over a limited range of values such that the temporal filter bandwidth is larger than the spatial bandwidth. This property is shared by motion sensitive cells in the primary visual cortex of the cat, which are known to be direction selective and are tuned to spacetime frequency orientations. We conjecture that these cells have larger temporal bandwidth relative to their spatial bandwidth because they compute velocity with maximum efficiency, that is, with a minimum motion uncertainty.     

Location coding by opponent neural populations in the auditory cortex

Although the auditory cortex plays a necessary role in sound localization, physiological investigations in the cortex reveal inhomogeneous sampling of auditory space that is difficult to reconcile with localization behavior under the assumption of local spatial coding. Most neurons respond maximally to sounds located far to the left or right side, with few neurons tuned to the frontal midline. Paradoxically, psychophysical studies show optimal spatial acuity across the frontal midline. In this paper, we revisit the problem of inhomogeneous spatial sampling in three fields of cat auditory cortex. In each field, we confirm that neural responses tend to be greatest for lateral positions, but show the greatest modulation for near-midline source locations. Moreover, identification of source locations based on cortical responses shows sharp discrimination of left from right but relatively inaccurate discrimination of locations within each half of space. Motivated by these findings, we explore an opponent-process theory in which sound-source locations are represented by differences in the activity of two broadly tuned channels formed by contra- and ipsilaterally preferring neurons. Finally, we demonstrate a simple model, based on spike-count differences across cortical populations, that provides bias-free, level-invariant localization—and thus also a solution to the “binding problem” of associating spatial information with other nonspatial attributes of sounds.     

Input-Dependent Frequency Modulation of Cortical Gamma Oscillations Shapes Spatial Synchronization and Enables Phase Coding

Fine-scale temporal organization of cortical activity in the gamma range (∼25–80Hz) may play a significant role in information processing, for example by neural grouping (‘binding’) and phase coding. Recent experimental studies have shown that the precise frequency of gamma oscillations varies with input drive (e.g. visual contrast) and that it can differ among nearby cortical locations. This has challenged theories assuming widespread gamma synchronization at a fixed common frequency. In the present study, we investigated which principles govern gamma synchronization in the presence of input-dependent frequency modulations and whether they are detrimental for meaningful input-dependent gamma-mediated temporal organization. To this aim, we constructed a biophysically realistic excitatory-inhibitory network able to express different oscillation frequencies at nearby spatial locations. Similarly to cortical networks, the model was topographically organized with spatially local connectivity and spatially-varying input drive. We analyzed gamma synchronization with respect to phase-locking, phase-relations and frequency differences, and quantified the stimulus-related information represented by gamma phase and frequency. By stepwise simplification of our models, we found that the gamma-mediated temporal organization could be reduced to basic synchronization principles of weakly coupled oscillators, where input drive determines the intrinsic (natural) frequency of oscillators. The gamma phase-locking, the precise phase relation and the emergent (measurable) frequencies were determined by two principal factors: the detuning (intrinsic frequency difference, i.e. local input difference) and the coupling strength. In addition to frequency coding, gamma phase contained complementary stimulus information. Crucially, the phase code reflected input differences, but not the absolute input level. This property of relative input-to-phase conversion, contrasting with latency codes or slower oscillation phase codes, may resolve conflicting experimental observations on gamma phase coding. Our modeling results offer clear testable experimental predictions. We conclude that input-dependency of gamma frequencies could be essential rather than detrimental for meaningful gamma-mediated temporal organization of cortical activity.     

Effects of quadrat size and data measurement on the detection of boundaries

Abstract. With sampled field data, the accuracy of delineation of ecotones is directly related to the resolution of the data (i.e. spatial and measurement type) and to the edge detection algorithm used. In the present study the reliability is investigated of an edge detection algorithm (lattice-wombling) to delimit vegetation boundaries when different spatial resolutions (quadrat sizes) are used. To quantify whether the edge detection algorithm is robust, it was applied to data from woody species in a second-growth woodland measured at different spatial resolutions with different data types. Boundaries were found at similar locations using any reasonable quadrat size (ca. 200 m²) but there were some slight differences when using different vegetation measures (density versus presence/absence data).     

Human vision combines oriented filters to compute edges.

The experiments examined the perceived spatial structure of plaid patterns, composed of two or three sinusoidal gratings of the same spatial frequency, superimposed at different orientations. Perceived structure corresponded well with the pattern of zero crossings in the output of a circular spatial filter applied to the image. This lends some support to Marr & Hildreth's (Proc. R. Soc. Lond. B 207, 187 (1980)) theory of edge detection as a model for human vision, but with a very different implementation. The perceived structure of two-component plaids was distorted by prior exposure to a masking or adapting grating, in a way that was perceptually equivalent to reducing the contrast of one of the plaid components. This was confirmed by finding that the plaid distortion could be nulled by increasing the contrast of the masked or adapted component. A corresponding reduction of perceived contrast for single gratings was observed after adaptation and in some masking conditions. I propose the outlines of a model for edge finding in human vision. The plaid components are processed through cortical, orientation-selective filters that are subject to attenuation by forward masking and adaptation. The outputs of these oriented filters are then linearly summed to emulate circular filtering, and zero crossings (zcs) in the combined output are used to determine edge locations. Masking or adapting to a grating attenuates some oriented filters more than others, and although this changes only the effective contrast of the components, it results in a geometric distortion at the zc level after different filters have been combined. The orientation of zcs may not correspond at all with the orientation of Fourier components, but they are correctly predicted by this two-stage model. The oriented filters are not 'orientation detectors', but are precursors to a more subtle stage that locates and represents spatial features.     

Pre-attentive segmentation in the primary visual cortex

The activities of neurons in primary visual cortex have been shown to be significantly influenced by stimuli outside their classical receptive fields. We propose that these contextual influences serve pre-attentive visual segmentation by causing relatively higher neural responses to important or conspicuous image locations, making them more salient for perceptual pop-out. These locations include boundaries between regions, smooth contours, and pop-out targets against backgrounds. The mark of these locations is the breakdown of spatial homogeneity in the input. for instance, at the border between two texture regions of equal mean luminance. This breakdown causes changes in contextual influences, often resulting in higher responses at the border than at surrounding locations. This proposal is implemented in a biologically based model of VI in which contextual influences are mediated by intra-cortical horizontal connections. The behavior of the model is demonstrated using examples of texture segmentation, figure-ground segregation, target-distractor asymmetry, and contour enhancement, and is compared with psychophysical and physiological data. The model predicts (1) how neural responses should be tuned to the orientation of nearby texture borders, (2) a set of qualitative constraints on the structure of the intracortical connections, and (3) stimulus-dependent biases in estimating the locations of the region borders by pre-attentive vision.     

Assessing Greater Sage-Grouse Selection of Brood-Rearing Habitat Using Remotely-Sensed Imagery: Can Readily Available High-Resolution Imagery Be Used to Identify Brood-Rearing Habitat Across a Broad Landscape?

Greater sage-grouse populations have decreased steadily since European settlement in western North America. Reduced availability of brood-rearing habitat has been identified as a limiting factor for many populations. We used radio-telemetry to acquire locations of sage-grouse broods from 1998 to 2012 in Strawberry Valley, Utah. Using these locations and remotely-sensed NAIP (National Agricultural Imagery Program) imagery, we 1) determined which characteristics of brood-rearing habitat could be used in widely available, high resolution imagery 2) assessed the spatial extent at which sage-grouse selected brood-rearing habitat, and 3) created a predictive habitat model to identify areas of preferred brood-rearing habitat. We used AIC model selection to evaluate support for a list of variables derived from remotely-sensed imagery. We examined the relationship of these explanatory variables at three spatial extents (45, 200, and 795 meter radii). Our top model included 10 variables (percent shrub, percent grass, percent tree, percent paved road, percent riparian, meters of sage/tree edge, meters of riparian/tree edge, distance to tree, distance to transmission lines, and distance to permanent structures). Variables from each spatial extent were represented in our top model with the majority being associated with the larger (795 meter) spatial extent. When applied to our study area, our top model predicted 75% of naïve brood locations suggesting reasonable success using this method and widely available NAIP imagery. We encourage application of our methodology to other sage-grouse populations and species of conservation concern.     

Stereo disparity computation using Gabor filters

A solution to the correspondence problem for stereopsis is proposed using the differences in the complex phase of local spatial frequency components. One-dimensional spatial Gabor filters (Gabor 1946; Marcelja 1980), at different positions and spatial frequencies are convolved with each member of a stereo pair. The difference between the complex phase at corresponding points in the two images is used to find the stereo disparity. Disparity values are combined across spatial frequencies for each image location. Three-dimensional depth maps have been computed from real images under standard lighting conditions, as well as from random-dot stereograms (Julesz 1971). The algorithm can discriminate disparities significantly smaller than the width of a pixel. It is possible that a similar mechanism might be used in the human visual system.     

A binocular model for the simple cell

We authors propose a mathematical model for simple cell binocular response. It comprises two Gabor-type receptive fields (RF) having the same RF center, preferred spatial frequency, and preferred orientation. The model integrates the equally weighted signals from both eyes and performs a threshold operation. Poggio and Fischer (1977) classified binocular disparity cells in the striate cortex into four groups: tuned excitatory (TE), tuned inhibitory (TI), near, and far cells. They also found that most of the TE cells are ocularly balanced and that the other three types are usually unbalanced. This model can imitate these four types of disparity sensitivities and their ocular dominance tendency. We perform model fittings to Poggio's data using the “simulated annealing” method and discuss parameter dependence of the model's response. The model can also respond with exceptional disparity sensitivity: i.e., flat type, alternating type, and intermediate type.     

Landscape structure and boundary effects determine the fate of mutations occurring during range expansions

The interplay between the spatial dynamics of range expansion and evolutionary processes is receiving considerable attention. Recent theory has demonstrated that mutations occurring towards the front of a spatially expanding population can sometimes 'surf' to high frequency and spatial extent. Here, we extend this work to consider how the fate of a novel mutation is influenced by where and when it occurs. Specifically, we are interested in establishing how the origin of a mutation relative to a habitat edge influences its dynamics, and in understanding how this is mediated by the behaviour of individuals at those boundaries. Using a coupled-map lattice model, we demonstrate that the survival probability, abundance and spatial extent of surviving mutants can depend on their origin. An edge effect is often observed and can be quite different both qualitatively and quantitatively depending on the behavioural rules assumed. Mutations, especially those that are deleterious, that arise at a habitat edge with reflective boundary conditions can be many more times likely to survive for substantial periods of time than those that arise away from the edge. Conversely, with absorbing boundary conditions, their survival is greater when they arise well away from the edge. Our results clearly illustrate that landscape structure, habitat edges and boundary conditions have a considerable influence on the likely fate of mutations that occur during a period of range expansion.