The direction of retinal motion facilitates binocular stereopsis.

Visual information from binocular disparity and from relative motion provide information about three-dimensional structure and layout of the world. Although the mechanisms that process these cues have typically been studied independently, there is now a substantial body of evidence that suggests that they interact in the visual pathway. This paper investigates one advantage of such an interaction: whether retinal motion can be used as a matching constraint in the binocular correspondence process. Stimuli that contained identical disparity and motion signals but which differed in their fine-scale correlation were created to establish whether the direction, or the speed, of motion could enhance performance in a psychophysical task in which binocular matching is a limiting factor. The results of these experiments provide clear evidence that different directions of motion, but not different speeds, are processed separately in stereopsis. The results fit well with properties of neurons early in the cortical visual pathway which are thought to be involved in determining local matches between features in the two eyes'' images.     

Integration of Motion Responses Underlying Directional Motion Anisotropy in Human Early Visual Cortical Areas

Recent imaging studies have reported directional motion biases in human visual cortex when perceiving moving random dot patterns. It has been hypothesized that these biases occur as a result of the integration of motion detector activation along the path of motion in visual cortex. In this study we investigate the nature of such motion integration with functional MRI (fMRI) using different motion stimuli. Three types of moving random dot stimuli were presented, showing either coherent motion, motion with spatial decorrelations or motion with temporal decorrelations. The results from the coherent motion stimulus reproduced the centripetal and centrifugal directional motion biases in V1, V2 and V3 as previously reported. The temporally decorrelated motion stimulus resulted in both centripetal and centrifugal biases similar to coherent motion. In contrast, the spatially decorrelated motion stimulus resulted in small directional motion biases that were only present in parts of visual cortex coding for higher eccentricities of the visual field. In combination with previous results, these findings indicate that biased motion responses in early visual cortical areas most likely depend on the spatial integration of a simultaneously activated motion detector chain.     

Transitions between central and peripheral vision create spatial/temporal distortions: a hypothesis concerning the perceived break of the curveball

Background

The human visual system does not treat all parts of an image equally: the central segments of an image, which fall on the fovea, are processed with a higher resolution than the segments that fall in the visual periphery. Even though the differences between foveal and peripheral resolution are large, these differences do not usually disrupt our perception of seamless visual space. Here we examine a motion stimulus in which the shift from foveal to peripheral viewing creates a dramatic spatial/temporal discontinuity.

Methodology/Principal Findings

The stimulus consists of a descending disk (global motion) with an internal moving grating (local motion). When observers view the disk centrally, they perceive both global and local motion (i.e., observers see the disk''s vertical descent and the internal spinning). When observers view the disk peripherally, the internal portion appears stationary, and the disk appears to descend at an angle. The angle of perceived descent increases as the observer views the stimulus from further in the periphery. We examine the first- and second-order information content in the display with the use of a three-dimensional Fourier analysis and show how our results can be used to describe perceived spatial/temporal discontinuities in real-world situations.

Conclusions/Significance

The perceived shift of the disk''s direction in the periphery is consistent with a model in which foveal processing separates first- and second-order motion information while peripheral processing integrates first- and second-order motion information. We argue that the perceived distortion may influence real-world visual observations. To this end, we present a hypothesis and analysis of the perception of the curveball and rising fastball in the sport of baseball. The curveball is a physically measurable phenomenon: the imbalance of forces created by the ball''s spin causes the ball to deviate from a straight line and to follow a smooth parabolic path. However, the curveball is also a perceptual puzzle because batters often report that the flight of the ball undergoes a dramatic and nearly discontinuous shift in position as the ball nears home plate. We suggest that the perception of a discontinuous shift in position results from differences between foveal and peripheral processing.     

Integration of Sensory and Reward Information during Perceptual Decision-Making in Lateral Intraparietal Cortex (LIP) of the Macaque Monkey

Single neurons in cortical area LIP are known to carry information relevant to both sensory and value-based decisions that are reported by eye movements. It is not known, however, how sensory and value information are combined in LIP when individual decisions must be based on a combination of these variables. To investigate this issue, we conducted behavioral and electrophysiological experiments in rhesus monkeys during performance of a two-alternative, forced-choice discrimination of motion direction (sensory component). Monkeys reported each decision by making an eye movement to one of two visual targets associated with the two possible directions of motion. We introduced choice biases to the monkeys'' decision process (value component) by randomly interleaving balanced reward conditions (equal reward value for the two choices) with unbalanced conditions (one alternative worth twice as much as the other). The monkeys'' behavior, as well as that of most LIP neurons, reflected the influence of all relevant variables: the strength of the sensory information, the value of the target in the neuron''s response field, and the value of the target outside the response field. Overall, detailed analysis and computer simulation reveal that our data are consistent with a two-stage drift diffusion model proposed by Diederich and Bussmeyer [1] for the effect of payoffs in the context of sensory discrimination tasks. Initial processing of payoff information strongly influences the starting point for the accumulation of sensory evidence, while exerting little if any effect on the rate of accumulation of sensory evidence.     

Self versus Environment Motion in Postural Control

To stabilize our position in space we use visual information as well as non-visual physical motion cues. However, visual cues can be ambiguous: visually perceived motion may be caused by self-movement, movement of the environment, or both. The nervous system must combine the ambiguous visual cues with noisy physical motion cues to resolve this ambiguity and control our body posture. Here we have developed a Bayesian model that formalizes how the nervous system could solve this problem. In this model, the nervous system combines the sensory cues to estimate the movement of the body. We analytically demonstrate that, as long as visual stimulation is fast in comparison to the uncertainty in our perception of body movement, the optimal strategy is to weight visually perceived movement velocities proportional to a power law. We find that this model accounts for the nonlinear influence of experimentally induced visual motion on human postural behavior both in our data and in previously published results.     

Tuning Properties of MT and MSTd and Divisive Interactions for Eye-Movement Compensation

The primate brain intelligently processes visual information from the world as the eyes move constantly. The brain must take into account visual motion induced by eye movements, so that visual information about the outside world can be recovered. Certain neurons in the dorsal part of monkey medial superior temporal area (MSTd) play an important role in integrating information about eye movements and visual motion. When a monkey tracks a moving target with its eyes, these neurons respond to visual motion as well as to smooth pursuit eye movements. Furthermore, the responses of some MSTd neurons to the motion of objects in the world are very similar during pursuit and during fixation, even though the visual information on the retina is altered by the pursuit eye movement. We call these neurons compensatory pursuit neurons. In this study we develop a computational model of MSTd compensatory pursuit neurons based on physiological data from single unit studies. Our model MSTd neurons can simulate the velocity tuning of monkey MSTd neurons. The model MSTd neurons also show the pursuit compensation property. We find that pursuit compensation can be achieved by divisive interaction between signals coding eye movements and signals coding visual motion. The model generates two implications that can be tested in future experiments: (1) compensatory pursuit neurons in MSTd should have the same direction preference for pursuit and retinal visual motion; (2) there should be non-compensatory pursuit neurons that show opposite preferred directions of pursuit and retinal visual motion.     

Nonaccidental properties underlie shape recognition in Mammalian and nonmammalian vision

An infinite number of 2D patterns on the retina can correspond to a single 3D object. How do visual systems resolve this ill-posed problem and recognize objects from only a few 2D retinal projections in varied exposure conditions? Theories of object recognition rely on the nonaccidental statistics of edge properties, mainly symmetry, collinearity, curvilinearity, and cotermination. These statistics are determined by the image-formation process (i.e., the 2D retinal projection of a 3D object ); their existence under a range of viewpoints enables viewpoint-invariant recognition. An important question in behavioral biology is whether the visual systems of nonmammalian animals have also evolved biases to utilize nonaccidental statistics . Here, we trained humans and pigeons to recognize four shapes. With the Bubbles technique, we determined which stimulus properties both species used to recognize the shapes. Both humans and pigeons used cotermination, the most diagnostic nonaccidental property of real-world objects, despite evidence from a model computer observer that cotermination was not the most diagnostic pictorial information in this particular task. This result reveals that a nonmammalian visual system that is different anatomically from the human visual system is also biased to recognize objects from nonaccidental statistics.     

Relating Lateralization of Eye Use to Body Motion in the Avoidance Behavior of the Chameleon (Chamaeleo chameleon)

Lateralization is mostly analyzed for single traits, but seldom for two or more traits while performing a given task (e.g. object manipulation). We examined lateralization in eye use and in body motion that co-occur during avoidance behaviour of the common chameleon, Chamaeleo chameleon. A chameleon facing a moving threat smoothly repositions its body on the side of its perch distal to the threat, to minimize its visual exposure. We previously demonstrated that during the response (i) eye use and body motion were, each, lateralized at the tested group level (N = 26), (ii) in body motion, we observed two similar-sized sub-groups, one exhibiting a greater reduction in body exposure to threat approaching from the left and one – to threat approaching from the right (left- and right-biased subgroups), (iii) the left-biased sub-group exhibited weak lateralization of body exposure under binocular threat viewing and none under monocular viewing while the right-biased sub-group exhibited strong lateralization under both monocular and binocular threat viewing. In avoidance, how is eye use related to body motion at the entire group and at the sub-group levels? We demonstrate that (i) in the left-biased sub-group, eye use is not lateralized, (ii) in the right-biased sub-group, eye use is lateralized under binocular, but not monocular viewing of the threat, (iii) the dominance of the right-biased sub-group determines the lateralization of the entire group tested. We conclude that in chameleons, patterns of lateralization of visual function and body motion are inter-related at a subtle level. Presently, the patterns cannot be compared with humans'' or related to the unique visual system of chameleons, with highly independent eye movements, complete optic nerve decussation and relatively few inter-hemispheric commissures. We present a model to explain the possible inter-hemispheric differences in dominance in chameleons'' visual control of body motion during avoidance.     

Slow Feature Analysis on Retinal Waves Leads to V1 Complex Cells

The developing visual system of many mammalian species is partially structured and organized even before the onset of vision. Spontaneous neural activity, which spreads in waves across the retina, has been suggested to play a major role in these prenatal structuring processes. Recently, it has been shown that when employing an efficient coding strategy, such as sparse coding, these retinal activity patterns lead to basis functions that resemble optimal stimuli of simple cells in primary visual cortex (V1). Here we present the results of applying a coding strategy that optimizes for temporal slowness, namely Slow Feature Analysis (SFA), to a biologically plausible model of retinal waves. Previously, SFA has been successfully applied to model parts of the visual system, most notably in reproducing a rich set of complex-cell features by training SFA with quasi-natural image sequences. In the present work, we obtain SFA units that share a number of properties with cortical complex-cells by training on simulated retinal waves. The emergence of two distinct properties of the SFA units (phase invariance and orientation tuning) is thoroughly investigated via control experiments and mathematical analysis of the input-output functions found by SFA. The results support the idea that retinal waves share relevant temporal and spatial properties with natural visual input. Hence, retinal waves seem suitable training stimuli to learn invariances and thereby shape the developing early visual system such that it is best prepared for coding input from the natural world.     

A Bayesian model of stereopsis depth and motion direction discrimination

 The extraction of stereoscopic depth from retinal disparity, and motion direction from two-frame kinematograms, requires the solution of a correspondence problem. In previous psychophysical work [Read and Eagle (2000) Vision Res 40: 3345–3358], we compared the performance of the human stereopsis and motion systems with correlated and anti-correlated stimuli. We found that, although the two systems performed similarly for narrow-band stimuli, broad-band anti-correlated kinematograms produced a strong perception of reversed motion, whereas the stereograms appeared merely rivalrous. I now model these psychophysical data with a computational model of the correspondence problem based on the known properties of visual cortical cells. Noisy retinal images are filtered through a set of Fourier channels tuned to different spatial frequencies and orientations. Within each channel, a Bayesian analysis incorporating a prior preference for small disparities is used to assess the probability of each possible match. Finally, information from the different channels is combined to arrive at a judgement of stimulus disparity. Each model system – stereopsis and motion – has two free parameters: the amount of noise they are subject to, and the strength of their preference for small disparities. By adjusting these parameters independently for each system, qualitative matches are produced to psychophysical data, for both correlated and anti-correlated stimuli, across a range of spatial frequency and orientation bandwidths. The motion model is found to require much higher noise levels and a weaker preference for small disparities. This makes the motion model more tolerant of poor-quality reverse-direction false matches encountered with anti-correlated stimuli, matching the strong perception of reversed motion that humans experience with these stimuli. In contrast, the lower noise level and tighter prior preference used with the stereopsis model means that it performs close to chance with anti-correlated stimuli, in accordance with human psychophysics. Thus, the key features of the experimental data can be reproduced assuming that the motion system experiences more effective noise than the stereoscopy system and imposes a less stringent preference for small disparities. Received: 2 March 2001 / Accepted in revised form: 5 July 2001     

A model for the detection of moving targets in visual clutter inspired by insect physiology

We present a computational model for target discrimination based on intracellular recordings from neurons in the fly visual system. Determining how insects detect and track small moving features, often against cluttered moving backgrounds, is an intriguing challenge, both from a physiological and a computational perspective. Previous research has characterized higher-order neurons within the fly brain, known as 'small target motion detectors' (STMD), that respond robustly to moving features, even when the velocity of the target is matched to the background (i.e. with no relative motion cues). We recorded from intermediate-order neurons in the fly visual system that are well suited as a component along the target detection pathway. This full-wave rectifying, transient cell (RTC) reveals independent adaptation to luminance changes of opposite signs (suggesting separate ON and OFF channels) and fast adaptive temporal mechanisms, similar to other cell types previously described. From this physiological data we have created a numerical model for target discrimination. This model includes nonlinear filtering based on the fly optics, the photoreceptors, the 1(st) order interneurons (Large Monopolar Cells), and the newly derived parameters for the RTC. We show that our RTC-based target detection model is well matched to properties described for the STMDs, such as contrast sensitivity, height tuning and velocity tuning. The model output shows that the spatiotemporal profile of small targets is sufficiently rare within natural scene imagery to allow our highly nonlinear 'matched filter' to successfully detect most targets from the background. Importantly, this model can explain this type of feature discrimination without the need for relative motion cues.     

The Gaussian derivative model for spatial-temporal vision: I. Cortical model

How do we see the motion of objects as well as their shapes? The Gaussian Derivative (GD) spatial model is extended to time to help answer this question. The GD spatio-temporal model requires only two numbers to describe the complete three-dimensional space-time shapes of individual receptive fields in primate visual cortex. These two numbers are the derivative numbers along the respective spatial and temporal principal axes of a given receptive field. Nine transformation parameters allow for a standard geometric association of these intrinsic axes with the extrinsic environment. The GD spatio-temporal model describes in one framework the following properties of primate simple cell fields: motion properties, number of lobes in space-time, spatial orientation. location, and size. A discrete difference-of-offset-Gaussians (DOOG) model provides a plausible physiological mechanism to form GD-like model fields in both space and time. The GD model hypothesizes that receptive fields at the first stage of processing in the visual cortex approximate 'derivative analyzers' that estimate local spatial and temporal derivatives of the intensity profile in the visual environment. The receptive fields as modeled provide operators that can allow later stages of processing in either a biological or machine vision system to estimate the motion as well as the shapes of objects in the environment.     

Detecting binocular 3D motion in static 3D noise: no effect of viewing distance

Relative binocular disparity cannot tell us the absolute 3D shape of an object, nor the 3D trajectory of its motion, unless the visual system has independent access to how far away the object is at any moment. Indeed, as the viewing distance is changed, the same disparate retinal motions will correspond to very different real 3D trajectories. In this paper we were interested in whether binocular 3D motion detection is affected by viewing distance. A visual search task was used, in which the observer is asked to detect a target dot, moving in 3D, amidst 3D stationary distractor dots. We found that distance does not affect detection performance. Motion-in-depth is consistently harder to detect than the equivalent lateral motion, for all viewing distances. For a constant retinal motion with both lateral and motion-in-depth components, detection performance is constant despite variations in viewing distance that produce large changes in the direction of the 3D trajectory. We conclude that binocular 3D motion detection relies on retinal, not absolute, visual signals.     

Task-specific impairments and enhancements induced by magnetic stimulation of human visual area V5.

Transcranial magnetic stimulation (TMS) can be used to simulate the effects of highly circumscribed brain damage permanently present in some neuropsychological patients, by reversibly disrupting the normal functioning of the cortical area to which it is applied. By using TMS we attempted to recreate deficits similar to those reported in a motion-blind patient and to assess the specificity of deficits when TMS is applied over human area V5. We used six visual search tasks and showed that subjects were impaired in a motion but not a form ''pop-out'' task when TMS was applied over V5. When motion was present, but irrelevant, or when attention to colour and form were required, TMS applied to V5 enhanced performance. When attention to motion was required in a motion-form conjunction search task, irrespective of whether the target was moving or stationary, TMS disrupted performance. These data suggest that attention to different visual attributes involves mutual inhibition between different extrastriate visual areas.     

Stereoscopic subsystems for position in depth and for motion in depth.

We describe psychophysical evidence that the human visual system contains information-processing channels for motion in depth in addition to those for position in depth. These motion-in-depth channels include some that are selectively sensitive to the relative velocities of the left and right retinal images. We propose that the visual pathway contains stereoscopic (cyclopean) motion filters that respond to only a narrow range of the directions of motion in depth. Turning to the single-neuron level we report that, in addition to neurons turned to position to depth, cat visual cortex contains neurons that emphasize information about the direction of motion at the expense of positional information. We describe psychophysical evidence for the existence of channels that are sensitive to change size, and are separate from the channels both for motion and for flicker. These changing-size channels respond independently of whether the stimulus is a bright square on a dark ground or a dark square on a bright ground. At the physiological level we report single neurons in cat visual cortex that respond selectively to increasing or to decreasing size independently of the sign of stimulus contrast. Adaptation to a changing-size stimulus produces two separable after-effects: an illusion of changing size, and an illusion of motion in depth. These after-effects have different decay time constants. We propose a psychophysical model in which changing-size filters feed a motion-in-depth stage, and suppose that the motion-in-depth after-effect is due to activity at the motion-in-depth stage, while the changing-size after-effect is due to to activity at the changing-size and more peripheral stages. The motion-in-depth after-effect can be cancelled either by a changing-size test stimulus or by relative motion of the left and right retinal images. Opposition of these two cues can also cancel the impression of motion in depth produced by the adapting stimulus. These findings link the stereoscopic (cyclopean) motion filters and the changing-size filters: both feed the same motion-in-depth stage.     

On the role of medial geometry in human vision.

A key challenge underlying theories of vision is how the spatially restricted, retinotopically represented feature analysis can be integrated to form abstract, coordinate-free object models. A resolution likely depends on the use of intermediate-level representations which can on the one hand be populated by local features and on the other hand be used as atomic units underlying the formation of, and interaction with, object hypotheses. The precise structure of this intermediate representation derives from the varied requirements of a range of visual tasks which motivate a significant role for incorporating a geometry of visual form. The need to integrate input from features capturing surface properties such as texture, shading, motion, color, etc., as well as from features capturing surface discontinuities such as silhouettes, T-junctions, etc., implies a geometry which captures both regional and boundary aspects. Curves, as a geometric model of boundaries, have been extensively used as an intermediate representation in computational, perceptual, and physiological studies, while the use of the medial axis (MA) has been popular mainly in computer vision as a geometric region-based model of the interior of closed boundaries. We extend the traditional model of the MA to represent images, where each MA segment represents a region of the image which we call a visual fragment. We present a unified theory of perceptual grouping and object recognition where through various sequences of transformations of the MA representation, visual fragments are grouped in various configurations to form object hypotheses, and are related to stored models. The mechanisms underlying both the computation and the transformation of the MA is a lateral wave propagation model. Recent psychophysical experiments depicting contrast sensitivity map peaks at the medial axes of stimuli, and experiments on perceptual filling-in, and brightness induction and modulation, are consistent with both the use of an MA representation and a propagation-based scheme. Also, recent neurophysiological recordings in V1 correlate with the MA hypothesis and a horizontal propagation scheme. This evidence supports a geometric computational paradigm for processing sensory data where both dynamic in-plane propagation and feedforward-feedback connections play an integral role.     

Human brain activity during illusory visual jitter as revealed by functional magnetic resonance imaging

One central problem in vision is how to compensate for retinal slip. A novel illusion (visual jitter) suggests the compensation mechanism is based solely on retinal motion. Adaptation to visual noise attenuates the motion signals used by the compensation stage, producing illusory jitter due to the undercompensation of retinal slip. Here, we investigated the neural substrate of retinal slip compensation during this illusion using high-field fMRI and retinotopic mapping in flattened cortical format. When jitter perception occurred, MR signal decreased in lower stages of the visual system but increased prominently in area MT+. In conclusion, visual areas as early as V1 are responsible for the adaptation stage, and MT+ is involved in the compensation stage. The present finding suggests the pathway from V1 to MT+ has an important role in stabilizing the visual world.     

Propagation of photon noise and information transfer in visual motion detection

The extraction of the direction of motion from the time varying retinal images is one of the most basic tasks any visual system is confronted with. However, retinal images are severely corrupted by photon noise, in particular at low light levels, thus limiting the performance of motion detection mechanisms of what sort so ever. Here, we study how photon noise propagates through an array of Reichardt-type motion detectors that are commonly believed to underlie fly motion vision. We provide closed-form analytical expressions of the signal and noise spectra at the output of such a motion detector array. We find that Reichardt detectors reveal favorable noise suppression in the frequency range where most of the signal power resides. Most notably, due to inherent adaptive properties, the transmitted information about stimulus velocity remains nearly constant over a large range of velocity entropies. Action editor: Matthew Wiener     

Identification of the nonlinear state-space dynamics of the action-perception cycle for visually induced postural sway

Human subjects standing in a sinusoidally moving visual environment display postural sway with characteristic dynamical properties. We analyzed the spatiotemporal properties of this sway in an experiment in which the frequency of the visual motion was varied. We found a constant gain near 1, which implies that the sway motion matches the spatial parameters of the visual motion for a large range of frequencies. A linear dynamical model with constant parameters was compared quantitatively with the data. Its failure to describe correctly the spatiotemporal properties of the system led us to consider adaptive and nonlinear models. To differentiate between possible alternative structures we directly fitted nonlinear differential equations to the sway and visual motion trajectories on a trial-by-trial basis. We found that the eigenfrequency of the fitted model adapts strongly to the visual motion frequency. The damping coefficient decreases with increasing frequency. This indicates that the system destabilizes its postural state in the inertial frame. This leads to a faster internal dynamics which is capable of synchronizing posture with fast-moving visual environments. Using an algorithm which allows the identification of essentially nonlinear terms of the dynamics we found small nonlinear contributions. These nonlinearities are not consistent with a limit-cycle dynamics, accounting for the robustness of the amplitude of postural sway against frequency variations. We interpret our results in terms of active generation of postural sway specified by sensory information. We derive also a number of conclusions for a behavior-oriented analysis of the postural system.     

Applicability of quadratic and threshold models to motion discrimination in the rabbit retina

Computational and behavioral studies suggest that visual motion discrimination is based on quadratic nonlinearities. This raises the question of whether the behavior of motion sensitive neurons early in the visual system is actually quadratic. Theoretical studies show that mechanisms proposed for retinal directional selectivity do not behave quadratically at high stimulus contrast. However, for low contrast stimuli, models for these mechanisms may be grouped into three categories: purely quadratic, quadratic accompanied by a rectification, and models mediated by a high level threshold. We discriminated between these alternatives by analyzing the extracellular responses of ON-OFF directionally selective ganglion cells of the rabbit retina to drifting periodic gratings. The data show that purely-quadratic or high-threshold systems do not account for the behavior of these cells. However, their behavior is consistent with a rectified-quadratic model.