Non-directional motion detectors can be used to mimic optic flow dependent behaviors

Insect navigational behaviors including obstacle avoidance, grazing landings, and visual odometry are dependent on the ability to estimate flight speed based only on visual cues. In honeybees, this visual estimate of speed is largely independent of both the direction of motion and the spatial frequency content of the image. Electrophysiological recordings from the motion-sensitive cells believed to underlie these behaviors have long supported spatio-temporally tuned correlation-type models of visual motion detection whose speed tuning changes as the spatial frequency of a stimulus is varied. The result is an apparent conflict between behavioral experiments and the electrophysiological and modeling data. In this article, we demonstrate that conventional correlation-type models are sufficient to reproduce some of the speed-dependent behaviors observed in honeybees when square wave gratings are used, contrary to the theoretical predictions. However, these models fail to match the behavioral observations for sinusoidal stimuli. Instead, we show that non-directional motion detectors, which underlie the correlation-based computation of directional motion, can be used to mimic these same behaviors even when narrowband gratings are used. The existence of such non-directional motion detectors is supported both anatomically and electrophysiologically, and they have been hypothesized to be critical in the Dipteran elementary motion detector (EMD) circuit.     

Speed tuning in elementary motion detectors of the correlation type

A prominent model of visual motion detection is the so-called correlation or Reichardt detector. Whereas this model can account for many properties of motion vision, from humans to insects (review, Borst and Egelhaaf 1989), it has been commonly assumed that this scheme of motion detection is not well suited to the measurement of image velocity. This is because the commonly used version of the model, which incorporates two unidirectional motion detectors with opposite preferred directions, produces a response which varies not only with the velocity of the image, but also with its spatial structure and contrast. On the other hand, information on image velocity can be crucial in various contexts, and a number of recent behavioural experiments suggest that insects do extract velocity for navigational purposes (review, Srinivasan et al. 1996). Here we show that other versions of the correlation model, which consists of a single unidirectional motion detector or incorporates two oppositely directed detectors with unequal sensitivities, produce responses which vary with image speed and display tuning curves that are substantially independent of the spatial structure of the image. This surprising feature suggests simple strategies of reducing ambiguities in the estimation of speed by using components of neural hardware that are already known to exist in the visual system. Received: 30 April 1998 / Accepted in revised form: 18 September 1998     

Computational modelling of motion detectors: responses to two-frame displays

Schemes for motion detection fall into two classes. Reichardt correlators compare spatial luminance patterns at two locations at different times; gradient detectors compare spatial and temporal luminance gradients. Both are candidate operators for biological and machine vision systems. A large body of perceptual data exists, defining the properties of motion detectors used by human observers, which can form a basis for determining which class of detector is appropriate for the human visual system. Plausible versions of each detector were implemented, and their responses to a variety of two-frame stimuli were computed. Results indicated that both detectors can predict most of the data, but on balance gradient detectors offer the best working hypothesis for motion detection by human observers. This conclusion is necessarily limited to the type of stimuli used, and may require modification in the light of responses to continuously moving stimuli.     

Evidence in support of the duplex theory of mechanoreception.

Psychophysical thresholds were measured as the frequency, size and duration of vibration on the thenar eminence was varied. The results supported the theory that there are two functionally distinct receptor systems in the skin for the perception of mechanical disturbances. Spatial and temporal summation were found for the detection of high- but not low-frequency stimuli. Furthermore, magnitude estimation data revealed that temporal summation occurs for the perception of high-frequency stimuli presented over a wide range of suprathershold values. On the other hand, temporal summation was absent for low frequency stimuli that were presented at 5, 10, 15, and 20 db above threshold.     

Spatio-temporal interpolation is accomplished by binocular form and motion mechanisms

Spatio-temporal interpolation describes the ability of the visual system to perceive shapes as whole figures (Gestalts), even if they are moving behind narrow apertures, so that only thin slices of them meet the eye at any given point in time. The interpolation process requires registration of the form slices, as well as perception of the shape's global motion, in order to reassemble the slices in the correct order. The commonly proposed mechanism is a spatio-temporal motion detector with a receptive field, for which spatial distance and temporal delays are interchangeable, and which has generally been regarded as monocular. Here we investigate separately the nature of the motion and the form detection involved in spatio-temporal interpolation, using dichoptic masking and interocular presentation tasks. The results clearly demonstrate that the associated mechanisms for both motion and form are binocular rather than monocular. Hence, we question the traditional view according to which spatio-temporal interpolation is achieved by monocular first-order motion-energy detectors in favour of models featuring binocular motion and form detection.     

Visual Benefits in Apparent Motion Displays: Automatically Driven Spatial and Temporal Anticipation Are Partially Dissociated

Many behaviourally relevant sensory events such as motion stimuli and speech have an intrinsic spatio-temporal structure. This will engage intentional and most likely unintentional (automatic) prediction mechanisms enhancing the perception of upcoming stimuli in the event stream. Here we sought to probe the anticipatory processes that are automatically driven by rhythmic input streams in terms of their spatial and temporal components. To this end, we employed an apparent visual motion paradigm testing the effects of pre-target motion on lateralized visual target discrimination. The motion stimuli either moved towards or away from peripheral target positions (valid vs. invalid spatial motion cueing) at a rhythmic or arrhythmic pace (valid vs. invalid temporal motion cueing). Crucially, we emphasized automatic motion-induced anticipatory processes by rendering the motion stimuli non-predictive of upcoming target position (by design) and task-irrelevant (by instruction), and by creating instead endogenous (orthogonal) expectations using symbolic cueing. Our data revealed that the apparent motion cues automatically engaged both spatial and temporal anticipatory processes, but that these processes were dissociated. We further found evidence for lateralisation of anticipatory temporal but not spatial processes. This indicates that distinct mechanisms may drive automatic spatial and temporal extrapolation of upcoming events from rhythmic event streams. This contrasts with previous findings that instead suggest an interaction between spatial and temporal attention processes when endogenously driven. Our results further highlight the need for isolating intentional from unintentional processes for better understanding the various anticipatory mechanisms engaged in processing behaviourally relevant stimuli with predictable spatio-temporal structure such as motion and speech.     

Are direction and speed coded independently by the visual system? Evidence from visual search.

Three visual search experiments examined whether motion is coded as two separate features, speed and direction. Increasing the heterogeneity of the directions in which stimuli moved disrupted detection of a target defined by speed (fast among medium and slow nontargets), suggesting that speed is coded integrally with direction. However, heterogeneity in speed did not disrupt detection of a target moving in a particular direction among nontargets with different directions. This suggests that direction is coded independently of speed. The apparent paradox raised by these contrasting conclusions is consistent with neurophysiological and computational models of motion-detection, which suggest that low-levels of the visual system contain direction-detectors insensitive to speed, while speed is coded at higher levels by detectors which are also sensitive to direction. Evidence consistent with the existence of the latter conjunction detectors was obtained in a final experiment which found search for a conjunction of speed and direction to be parallel.     

A descending contralateral directionally selective movement detector in the praying mantis <Emphasis Type="Italic">Tenodera aridifolia</Emphasis>

Extracellular recordings were made from a directionally selective neuron in the ventral nerve cord of mantises. The neuron’s preferred direction of motion was forward and upward over the compound eye contralateral to its axon at the cervical connective. The neuron was sensitive to wide-field motion stimuli, resistant to habituation, and showed transient excitation in response to light ON and OFF stimuli. Its responses to drifting gratings depended on the temporal frequency and contrast of the stimulus. These results suggest that the neuron receives input from correlation-type motion detectors.     

Visual detection of paradoxical motion in flies

Summary From psychophysics it is known that humans easily perceive motion in Fourier-stimuli in which dots are displaced coherently into one direction. Furthermore, motion can be extracted from Drift-balanced stimuli in which the dots on average have no distinct direction of motion, or even in paradox -motion stimuli where the dots are displaced opposite to the perceived direction of motion. Whereas Fourier-motion can be explained by very basic motion detectors and nonlinear preprocessing of the input can account for the detection of Drift-balanced motion, a hierarchical model with two layers of motion detectors was proposed to explain the perception of -motion. The well described visual system of the fly allows to investigate whether these complex motion stimuli can be detected in a comparatively simple brain.The detection of such motion stimuli was analyzed for various random-dot cinematograms with extracellular recordings from the motion-sensitive Hl-neuron in the third visual ganglion of the blowfly Calliphora erythrocephala. The results were compared to computer-simulations of a hierarchical model of motion detector networks.For Fourier- and Drift-balanced motion stimuli, the Hl-neuron responds directionally selective to the moving object, whereas for -motion stimuli, the preferred direction is given by the dot displacement. Assuming nonlinear preprocessing of the detector input, such as a half-wave rectification, elementary motion detectors of the correlation type can account for these results.Abbreviations EMD elementary motion detector     

Cardinal directions for visual optic flow.

As we move through our environment, the flow of deforming images on the retinae provides a rich source of information about the three-dimensional structure of the external world and how to navigate through it. Recent evidence from psychophysical [1] [2] [3] [4], electrophysiological [5] [6] [7] [8] [9] and imaging [10] [11] studies suggests that there are neurons in the primate visual system - in the medial superior temporal cortex - that are specialised to respond to this type of complex 'optic flow' motion. In principle, optic flow could be encoded by a small number of neural mechanisms tuned to 'cardinal directions', including radial and circular motion [12] [13]. There is little support for this idea at present, however, from either physiological [6] [7] or psychophysical [14] research. We have measured the sensitivity of human subjects for detection of motion and for discrimination of motion direction over a wide and densely sampled range of complex motions. Average sensitivity was higher for inward and outward radial movement and for both directions of rotation, consistent with the existence of detectors tuned to these four types of motion. Principle component analysis revealed two clear components, one for radial stimuli (outward and inward) and the other for circular stimuli (clockwise and counter-clock-wise). The results imply that the mechanisms that analyse optic flow in humans tend to be tuned to the cardinal axes of radial and rotational motion.     

A Bio-Inspired,Motion-Based Analysis of Crowd Behavior Attributes Relevance to Motion Transparency,Velocity Gradients,and Motion Patterns

The analysis of motion crowds is concerned with the detection of potential hazards for individuals of the crowd. Existing methods analyze the statistics of pixel motion to classify non-dangerous or dangerous behavior, to detect outlier motions, or to estimate the mean throughput of people for an image region. We suggest a biologically inspired model for the analysis of motion crowds that extracts motion features indicative for potential dangers in crowd behavior. Our model consists of stages for motion detection, integration, and pattern detection that model functions of the primate primary visual cortex area (V1), the middle temporal area (MT), and the medial superior temporal area (MST), respectively. This model allows for the processing of motion transparency, the appearance of multiple motions in the same visual region, in addition to processing opaque motion. We suggest that motion transparency helps to identify “danger zones” in motion crowds. For instance, motion transparency occurs in small exit passages during evacuation. However, motion transparency occurs also for non-dangerous crowd behavior when people move in opposite directions organized into separate lanes. Our analysis suggests: The combination of motion transparency and a slow motion speed can be used for labeling of candidate regions that contain dangerous behavior. In addition, locally detected decelerations or negative speed gradients of motions are a precursor of danger in crowd behavior as are globally detected motion patterns that show a contraction toward a single point. In sum, motion transparency, image speeds, motion patterns, and speed gradients extracted from visual motion in videos are important features to describe the behavioral state of a motion crowd.     

Colour and luminance interactions in the visual perception of motion

We sought to determine the extent to which red-green, colour-opponent mechanisms in the human visual system play a role in the perception of drifting luminance-modulated targets. Contrast sensitivity for the directional discrimination of drifting luminance-modulated (yellow-black) test sinusoids was measured following adaptation to isoluminant red-green sinusoids drifting in either the same or opposite direction. When the test and adapt stimuli drifted in the same direction, large sensitivity losses were evident at all test temporal frequencies employed (1-16 Hz). The magnitude of the loss was independent of temporal frequency. When adapt and test stimuli drifted in opposing directions, large sensitivity losses were evident at lower temporal frequencies (1-4 Hz) and declined with increasing temporal frequency. Control studies showed that this temporal-frequency-dependent effect could not reflect the activity of achromatic units. Our results provide evidence that chromatic mechanisms contribute to the perception of luminance-modulated motion targets drifting at speeds of up to at least 32 degrees s(-1). We argue that such mechanisms most probably lie within a parvocellular-dominated cortical visual pathway, sensitive to both chromatic and luminance modulation, but only weakly selective for the direction of stimulus motion.     

Propagation of photon noise and information transfer in visual motion detection

The extraction of the direction of motion from the time varying retinal images is one of the most basic tasks any visual system is confronted with. However, retinal images are severely corrupted by photon noise, in particular at low light levels, thus limiting the performance of motion detection mechanisms of what sort so ever. Here, we study how photon noise propagates through an array of Reichardt-type motion detectors that are commonly believed to underlie fly motion vision. We provide closed-form analytical expressions of the signal and noise spectra at the output of such a motion detector array. We find that Reichardt detectors reveal favorable noise suppression in the frequency range where most of the signal power resides. Most notably, due to inherent adaptive properties, the transmitted information about stimulus velocity remains nearly constant over a large range of velocity entropies. Action editor: Matthew Wiener     

Dynamic characteristics of spatial mechanisms coding contour structures

Psychophysical thresholds for the detection of luminance targets improve significantly when the targets are presented in a specific context of spatially separated, collinear inducing stimuli defining visual contours. This phenomenon is generally referred to as a special case of detection facilitation called spatial facilitation. Spatial facilitation has been observed with luminance-defined. achromatic stimuli on achromatic backgrounds as well as with targets and inducers defined by colour contrast. This paper reviews psychophysical results from detection experiments with human observers showing the conditions under which spatially separated contour inducers facilitate the detection of simultaneously presented target stimuli. The findings point towards two types of spatial mechanisms: (i) Short-range mechanisms that are sensitive to narrowly spaced stimuli of small size and, at distinct target locations, selective to the contrast polarity of targets and inducers. (ii) Long-range mechanisms that are triggered by longer stimuli, generate facilitation across wider spatial gaps between targets and inducers, and are insensitive to their contrast polarity. Spatial facilitation with chromatic stimuli requires a longer inducer exposure than spatial facilitation with achromatic stimuli, which is already fully effective at inducer exposures of 30 ms. This difference in temporal dynamics indicates some functional segregation between mechanisms for colour and luminance contrast in spatial coding. In general, spatially induced detection facilitation can to a large extent be explained by mechanisms involving from-short-to-long-range interactions between cortical detectors.     

Seeing objects in motion

This paper reports estimates of the conjoint spatiotemporal tuning functions of the neural mechanisms of the human vision system which detect image motion. The functions were derived from measurements of the minimum contrast necessary to detect the direction of drift of a sinusoidal grating, in the presence of phase-reversed masking gratings of various spatial and temporal frequencies. A mask of similar spatial and temporal frequencies to the test grating reduces sensitivity considerably, whereas one differing greatly in spatial or temporal frequency has little or no effect. The results show that for test gratings drifting at 8 Hz, the tuning function is bandpass in both space and time, peaked at the temporal and spatial frequency (SF) of the test (SFs were 0.1, 1 or 5 c deg-1; c represents cycles throughout). For a grating of 5 c deg-1 drifting at 0.3 Hz, the function is bandpass in space but lowpass in time. Fourier transform of the frequency results yields a function in space-time which we term the 'spatiotemporal receptive field'. For movement detectors (bandpass in space and time) the fields comprise alternating ridges of opposing polarity, elongated in space-time along the preferred velocity axis of the detector. We suggest that this organization explains how detectors analyse form and motion concurrently and accounts, at least in part, for a variety of perceptual phenomena, including summation, reduction of motion smear, metacontrast, stroboscopic motion and spatiotemporal interpolation.     

Detection of object motion by a fly neuron during simulated flight

Object detection on the basis of relative motion was investigated in the fly at the neuronal level. A representative of the figure detection cells (FD-cells), the FD1b-cell, was characterized with respect to its responses to optic flow which simulated the presence of an object during translatory flight. The figure detection cells reside in the fly's third visual neuropil and are believed to play a central role in mediating object-directed turning behaviour. The dynamical response properties as well as the mean response amplitudes of the FD1b-cell depend on the temporal frequency of object motion and on the presence or absence of background motion. The responses of the FD1b-cell to object motion during simulated translatory flight were compared to behavioural responses of the fly as obtained with identical stimuli in a previous study. The behavioural responses could only partly be explained on the basis of the FD1b-cell's responses. Further processing between the third visual neuropil and the final motor output has to be assumed which involves (1) facilitation of the object-induced responses during translatory background motion at moderate temporal frequencies, and (2) inhibition of the object-induced turning responses during translatory background motion at high temporal frequencies. Accepted: 9 October 1999     

Direction repulsion goes global

When viewing two superimposed, translating sets of dots moving in different directions, one overestimates direction difference. This phenomenon of direction repulsion is thought to be driven by inhibitory interactions between directionally tuned motion detectors. However, there is disagreement on where this occurs-at early stages of motion processing, when local motions are extracted; or at the later, global motion-processing stage following "pooling" of these local measures. These two stages of motion processing have been identified as occurring in area V1 and the human homolog of macaque MT/V5, respectively. We designed experiments in which local and global predictions of repulsion are pitted against one another. Our stimuli contained a target set of dots, moving at a uniform speed, superimposed on a "mixed-speed" distractor set. Because the perceived speed of a mixed-speed stimulus is equal to the dots' average speed, a global-processing account of direction repulsion predicts that repulsion magnitude induced by a mixed-speed distractor will be indistinguishable from that induced by a single-speed distractor moving at the same mean speed. This is exactly what we found. These results provide compelling evidence that global-motion interactions play a major role in driving direction repulsion.     

Motion: the long and short of it

Several authors have proposed that motion is analyzed by two separate processes: short-range and long-range. We claim that the differences between short-range and long-range motion phenomena are a direct consequence of the stimuli used in the two paradigms and are not evidence for the existence of two qualitatively different motion processes. We propose that a single style of motion analysis, similar to the well known Reichardt and Marr-Ullman motion detectors, underlies all motion phenomena. Although there are different detectors of this type specialized for different visual attributes (namely first-order and second-order stimuli), they all share the same mode of operation. We review the studies of second-order motion stimuli to show that they share the basic phenomena observed for first-order stimuli. The similarity across stimulus types suggests, not parallel streams of motion extraction, one short-range and passive and the other long-range and intelligent, but a concatenation of a common mode of initial motion extraction followed by a general inference process.     

Object Segmentation from Motion Discontinuities and Temporal Occlusions–A Biologically Inspired Model

Background

Optic flow is an important cue for object detection. Humans are able to perceive objects in a scene using only kinetic boundaries, and can perform the task even when other shape cues are not provided. These kinetic boundaries are characterized by the presence of motion discontinuities in a local neighbourhood. In addition, temporal occlusions appear along the boundaries as the object in front covers the background and the objects that are spatially behind it.

Methodology/Principal Findings

From a technical point of view, the detection of motion boundaries for segmentation based on optic flow is a difficult task. This is due to the problem that flow detected along such boundaries is generally not reliable. We propose a model derived from mechanisms found in visual areas V1, MT, and MSTl of human and primate cortex that achieves robust detection along motion boundaries. It includes two separate mechanisms for both the detection of motion discontinuities and of occlusion regions based on how neurons respond to spatial and temporal contrast, respectively. The mechanisms are embedded in a biologically inspired architecture that integrates information of different model components of the visual processing due to feedback connections. In particular, mutual interactions between the detection of motion discontinuities and temporal occlusions allow a considerable improvement of the kinetic boundary detection.

Conclusions/Significance

A new model is proposed that uses optic flow cues to detect motion discontinuities and object occlusion. We suggest that by combining these results for motion discontinuities and object occlusion, object segmentation within the model can be improved. This idea could also be applied in other models for object segmentation. In addition, we discuss how this model is related to neurophysiological findings. The model was successfully tested both with artificial and real sequences including self and object motion.     

Similar Mechanisms Underlie the Detection of Horizontal and Vertical Disparity Corrugations

Our aim was to compare sensitivity for horizontal and vertical disparity corrugations and to resolve whether these stimuli are processed by similar or radically different underlying mechanisms. We measure global disparity sensitivity as a function of carrier spatial frequency for equi-detectable carriers and found a similar optimal carrier relationship for vertical and horizontal stimuli. Sensitivity as a function of corrugation spatial frequency for stimuli of comparable spatial summation and composed of optimal, equi-detectable narrowband carriers did not significantly differ for vertical and horizontal stimuli. A small anisotropy was revealed when fixed, high contrast broadband carriers were used. In a separate discrimination-at-threshold experiment, multiple mechanisms of similar tuning were revealed to underlie the detection of both vertical and horizontal disparity corrugations. We conclude that the processing of the horizontal and vertical disparity corrugations occurs along similar lines.