Motion-based analysis of spatial patterns by the human visual system

BACKGROUND: It is known that the visibility of patterns presented through stationary multiple slits is significantly improved by pattern movements. This study investigated whether this spatiotemporal pattern interpolation is supported by motion mechanisms, as opposed to the general belief that the human visual cortex initially analyses spatial patterns independent of their movements. RESULTS: Psychophysical experiments showed that multislit viewing could not be ascribed to such motion-irrelevant factors as retinal painting by tracking eye movements or an increase in the number of views by pattern movements. Pattern perception was more strongly impaired by the masking noise moving in the same direction than by the noise moving in the opposite direction, which indicates the direction selectivity of the pattern interpolation mechanism. A direction-selective impairment of pattern perception by motion adaptation also indicates the direction selectivity of the interpolation mechanism. Finally, the map of effective spatial frequencies, estimated by a reverse-correlation technique, indicates observers' perception of higher spatial frequencies, the recovery of which is theoretically impossible without the aid of motion information. CONCLUSIONS: These results provide clear evidence against the notion of separate analysis of pattern and motion. The visual system uses motion mechanisms to integrate spatial pattern information along the trajectory of pattern movement in order to obtain clear perception of moving patterns. The pattern integration mechanism is likely to be direction-selective filtering by V1 simple cells, but the integration of the local pattern information into a global figure should be guided by a higher-order motion mechanism such as MT pattern cells.     

Stereoscopic and contrast-defined motion in human vision.

There is considerable evidence for the existence of a specialized mechanism in human vision for detecting moving contrast modulations and some evidence for a mechanism for detecting moving stereoscopic depth modulations. It is unclear whether a single second-order motion mechanism detects both types of stimulus or whether they are detected separately. We show that sensitivity to stereo-defined motion resembles that to contrast-defined motion in two important ways. First, when a missing-fundamental disparity waveform is moved in steps of 0.25 cycles, its perceived direction tends to reverse. This is a property of both luminance-defined and contrast-defined motion and is consistent with independent detection of motion at different spatial scales. Second, thresholds for detecting the direction of a smoothly drifting sinusoidal disparity modulation are much higher than those for detecting its orientation. This is a property of contrast-modulated gratings but not luminance-modulated gratings, for which the two thresholds are normally identical. The results suggest that stereo-defined and contrast-defined motion stimuli are detected either by a common mechanism or by separate mechanisms sharing a common principle of operation.     

The hierarchy of directional interactions in visual motion processing

It is well known that context influences our perception of visual motion direction. For example, spatial and temporal context manipulations can be used to induce two well-known motion illusions: direction repulsion and the direction after-effect (DAE). Both result in inaccurate perception of direction when a moving pattern is either superimposed on (direction repulsion), or presented following adaptation to (DAE), another pattern moving in a different direction. Remarkable similarities in tuning characteristics suggest that common processes underlie the two illusions. What is not clear, however, is whether the processes driving the two illusions are expressions of the same or different neural substrates. Here we report two experiments demonstrating that direction repulsion and the DAE are, in fact, expressions of different neural substrates. Our strategy was to use each of the illusions to create a distorted perceptual representation upon which the mechanisms generating the other illusion could potentially operate. We found that the processes mediating direction repulsion did indeed access the distorted perceptual representation induced by the DAE. Conversely, the DAE was unaffected by direction repulsion. Thus parallels in perceptual phenomenology do not necessarily imply common neural substrates. Our results also demonstrate that the neural processes driving the DAE occur at an earlier stage of motion processing than those underlying direction repulsion.     

Neural population representation hypothesis of visual flow and its illusory after effect in the brain: psychophysics,neurophysiology and computational approaches

The neural representation of motion aftereffects induced by various visual flows (translational, rotational, motion-in-depth, and translational transparent flows) was studied under the hypothesis that the imbalances in discharge activities would occur in favor in the direction opposite to the adapting stimulation in the monkey MST cells (cells in the medial superior temporal area) which can discriminate the mode (i.e., translational, rotational, or motion-in-depth) of the given flow. In single-unit recording experiments conducted on anaesthetized monkeys, we found that the rate of spontaneous discharge and the sensitivity to a test stimulus moving in the preferred direction decreased after receiving an adapting stimulation moving in the preferred direction, whereas they increased after receiving an adapting stimulation moving in the null direction. To consistently explain the bidirectional perception of a transparent visual flow and its unidirectional motion aftereffect by the same hypothesis, we need to assume the existence of two subtypes of MST D cells which show directionally selective responses to a translational flow: component cells and integration cells. Our physiological investigation revealed that the MST D cells could be divided into two types: one responded to a transparent flow by two peaks at the instances when the direction of one of the component flow matched the preferred direction of the cell, and the other responded by a single peak at the instance when the direction of the integrated motion matched the preferred direction. In psychophysical experiments on human subjects, we found evidence for the existence of component and integration representations in the human brain. To explain the different motion perceptions, i.e., two transparent flows during presentation of the flows and a single flow in the opposite direction to the integrated flows after stopping the flow stimuli, we suggest that the pattern-discrimination system can select the motion representation that is consistent with the perception of the pattern from two motion representations. We discuss the computational aspects related to the integration of component motion fields.     

Separable temporal metrics for time perception and anticipatory actions

Reliable estimates of time are essential for initiating interceptive actions at the right moment. However, our sense of time is surprisingly fallible. For instance, time perception can be distorted by prolonged exposure (adaptation) to movement. Here, we make use of this to determine if time perception and anticipatory actions rely on the same or on different temporal metrics. Consistent with previous reports, we find that the apparent duration of movement is mitigated by adaptation to more rapid motion, but is unchanged by adaptation to slower movement. By contrast, we find symmetrical effects of motion-adaptation on the timing of anticipatory interceptive actions, which are paralleled by changes in perceived speed for the adapted direction of motion. Our data thus reveal that anticipatory actions and perceived duration rely on different temporal metrics.     

Interocular motion combination for dichoptic moving stimuli

When gratings moving in different directions are presented separately to the two eyes, we typically perceive periods of the combination of motion in the two eyes as well as periods of one or the other monocular motions. To investigate whether such interocular motion combination is determined by the intersection-of-constraints (IOC) or vector average mechanism, we recorded both optokinetic nystagmus eye movements (OKN) and perception during dichoptic presentation of moving gratings and random-dot patterns with various differences of interocular motion direction. For moving gratings, OKN alternately tracks not only the direction of the two monocular motions but also the direction of their combined motion. The OKN in the combined motion direction is highly correlated with the perceived direction of combined motion; its velocity complies with the IOC rule rather than the vector average of the dichoptic motion stimuli. For moving random-dot patterns, both OKN and perceived motion alternate only between the directions of the two monocular motions. These results suggest that interocular motion combination in dichoptic gratings is determined by the IOC and depends on their form.     

Perceived direction of motion determined by adaptation to static binocular images

In Li and Atick's [1, 2] theory of efficient stereo coding, the two eyes' signals are transformed into uncorrelated binocular summation and difference signals, and gain control is applied to the summation and differencing channels to optimize their sensitivities. In natural vision, the optimal channel sensitivities vary from moment to moment, depending on the strengths of the summation and difference signals; these channels should therefore be separately adaptable, whereby a channel's sensitivity is reduced following overexposure to adaptation stimuli that selectively stimulate that channel. This predicts a remarkable effect of binocular adaptation on perceived direction of a dichoptic motion stimulus [3]. For this stimulus, the summation and difference signals move in opposite directions, so perceived motion direction (upward or downward) should depend on which of the two binocular channels is most strongly adapted, even if the adaptation stimuli are completely static. We confirmed this prediction: a single static dichoptic adaptation stimulus presented for less than 1 s can control perceived direction of a subsequently presented dichoptic motion stimulus. This is not predicted by any current model of motion perception and suggests that the visual cortex quickly adapts to the prevailing binocular image statistics to maximize information-coding efficiency.     

Human visual system integrates color signals along a motion trajectory

Whether fundamental visual attributes, such as color, motion, and shape, are analyzed separately in specialized pathways has been one of the central questions of visual neuroscience. Although recent studies have revealed various forms of cross-attribute interactions, including significant contributions of color signals to motion processing, it is still widely believed that color perception is relatively independent of motion processing. Here, we report a new color illusion, motion-induced color mixing, in which moving bars, the color of each of which alternates between two colors (e.g., red and green), are perceived as the mixed color (e.g., yellow) even though the two colors are never superimposed on the retina. The magnitude of color mixture is significantly stronger than that expected from direction-insensitive spatial integration of color signals. This illusion cannot be ascribed to optical image blurs, including those induced by chromatic aberration, or to involuntary eye movements of the observer. Our findings indicate that color signals are integrated not only at the same retinal location, but also along a motion trajectory. It is possible that this neural mechanism helps us to see veridical colors for moving objects by reducing motion blur, as in the case of luminance-based pattern perception.     

基于EMDs阵列和BP网络的视觉运动感知过程

以Ｒｅｉｃｈａｒｄｔ的相关型初级运动检测器阵列和Ｒｕｍｅｌｈａｒｔ的误差反传学习（ｌｅａｒｎｉｎｇｂｙｂａｃｋ－ｐｒｏｐａｇａｔｉｎｇｅｒｒｏｒｓ,ＢＰ）网络相结合构成了一个视觉运动感知神经网络,探讨了视觉运动信息的感知过程。试图从计算神经科学的观点来阐明从一推运动分量的检测到二维模式运动感知的神经原理,从而回答运动矢量在脑内如何表征。计算机仿真表明,在有监督学习的条件下,网络可以学会解决局城运动检测所带来的多义性问题,给出模式的真实朝向、运动方向和运动速度。     

The visual processing of motion-defined transparency

Our understanding of how the visual system processes motion transparency, the phenomenon by which multiple directions of motion are perceived to coexist in the same spatial region, has grown considerably in the past decade. There is compelling evidence that the process is driven by global-motion mechanisms. Consequently, although transparently moving surfaces are readily segmented over an extended space, the visual system cannot separate two motion signals that coexist in the same local region. A related issue is whether the visual system can detect transparently moving surfaces simultaneously or whether the component signals encounter a serial 'bottleneck' during their processing. Our initial results show that, at sufficiently short stimulus durations, observers cannot accurately detect two superimposed directions; yet they have no difficulty in detecting one pattern direction in noise, supporting the serial-bottleneck scenario. However, in a second experiment, the difference in performance between the two tasks disappears when the component patterns are segregated. This discrepancy between the processing of transparent and non-overlapping patterns may be a consequence of suppressed activity of global-motion mechanisms when the transparent surfaces are presented in the same depth plane. To test this explanation, we repeated our initial experiment while separating the motion components in depth. The marked improvement in performance leads us to conclude that transparent motion signals are represented simultaneously.     

Perceived motion in complementary afterimages: verification of a neural network theory

Steady fixation of a regular pattern like a bar grating or concentric circles leads to a complementary afterimage at pattern offset. The afterimage has the appearance of shimmering lines that are locally orthogonal to the orientations of the inducing image. Additionally, the afterimage includes motion running parallel to the orientation of the afterimage lines. We argue that this afterimage motion supports the existence of a cue to motion that is based on the spatial organization of oriented responses. This cue was previously proposed after analysis of a neural network model of visual perception. We test predictions of the model on various types of complementary afterimage inducing stimuli. When a contrast or size gradient is included in the inducing image, the afterimage motion moves toward the higher part of the gradient, in agreement with the model. Implications of this cue for computational and neurophysiological theories of motion perception are discussed.     

Dichoptic plaids may rival, but their motions can integrate

When the eyes view incompatible images, binocular rivalry usually results: image constituents in corresponding parts of the monocular visual fields are not perceived simultaneously. We asked naive undergraduates to view dichoptic, dioptic, and monoptic plaids. The dichoptic images evoked strong binocular rivalry when contrast was high, especially if the component gratings were set in motion. Nevertheless, the subjects' visual systems integrated the motion information across the two eyes, producing a unitary motion percept that did not reflect the image in either eye alone. By manipulating the relative spatial scale of the gratings, we affected how well the motion cohered: the results were remarkably similar between dichoptic and traditional dioptic plaids. By manipulating the relative speed of the gratings, we systematically affected the perceived direction of motion of the plaids; these results were also remarkably similar for dichoptic and dioptic plaids. Thus, the motion analysis of dichoptic and dioptic plaids is proceeding according to very similar rules, even though the dichoptic images are incompatible and evoke binocular rivalry.     

Bifurcation study of a neural field competition model with an application to perceptual switching in motion integration

Perceptual multistability is a phenomenon in which alternate interpretations of a fixed stimulus are perceived intermittently. Although correlates between activity in specific cortical areas and perception have been found, the complex patterns of activity and the underlying mechanisms that gate multistable perception are little understood. Here, we present a neural field competition model in which competing states are represented in a continuous feature space. Bifurcation analysis is used to describe the different types of complex spatio-temporal dynamics produced by the model in terms of several parameters and for different inputs. The dynamics of the model was then compared to human perception investigated psychophysically during long presentations of an ambiguous, multistable motion pattern known as the barberpole illusion. In order to do this, the model is operated in a parameter range where known physiological response properties are reproduced whilst also working close to bifurcation. The model accounts for characteristic behaviour from the psychophysical experiments in terms of the type of switching observed and changes in the rate of switching with respect to contrast. In this way, the modelling study sheds light on the underlying mechanisms that drive perceptual switching in different contrast regimes. The general approach presented is applicable to a broad range of perceptual competition problems in which spatial interactions play a role.     

Adaptive Changes of Spatial Characteristics of Hearing in the Dolphin Tursiops truncatus

Adaptive abilities of the dolphin auditory system at perception of tonal signals of different frequencies were studied under conditions of spatial uncertainty of their presentation in the noise field. It has been shown that a transition from the threshold to the suprathreshold detection is accompanied by adaptive changes of spatial characteristics of auditory perception: transformation of the one-lobe direction diagrams into the two-lobe ones is observed, as well as their broadening and a shift of the sensitivity maximum from the frontal direction to the side one. The dependence of direction characteristics on frequency of the perceived signal is lost.     

Neural adjustments to chromatic blur

The perception of blur in images can be strongly affected by prior adaptation to blurry images or by spatial induction from blurred surrounds. These contextual effects may play a role in calibrating visual responses for the spatial structure of luminance variations in images. We asked whether similar adjustments might also calibrate the visual system for spatial variations in color. Observers adjusted the amplitude spectra of luminance or chromatic images until they appeared correctly focused, and repeated these measurements either before or after adaptation to blurred or sharpened images or in the presence of blurred or sharpened surrounds. Prior adaptation induced large and distinct changes in perceived focus for both luminance and chromatic patterns, suggesting that luminance and chromatic mechanisms are both able to adjust to changes in the level of blur. However, judgments of focus were more variable for color, and unlike luminance there was little effect of surrounding spatial context on perceived blur. In additional measurements we explored the effects of adaptation on threshold contrast sensitivity for luminance and color. Adaptation to filtered noise with a 1/f spectrum characteristic of natural images strongly and selectively elevated thresholds at low spatial frequencies for both luminance and color, thus transforming the chromatic contrast sensitivity function from lowpass to nearly bandpass. These threshold changes were found to reflect interactions between different spatial scales that bias sensitivity against the lowest spatial grain in the image, and may reflect adaptation to different stimulus attributes than the attributes underlying judgments of image focus. Our results suggest that spatial sensitivity for variations in color can be strongly shaped by adaptation to the spatial structure of the stimulus, but point to dissociations in these visual adjustments both between luminance and color and different measures of spatial sensitivity.     

A model of motion adaptation and motion after-effects based upon principal component regression

A computational model to help explain effects of adaptation to moving signals is compared with established energy (linear regression) models of motion detection. The proposed model assumes that processed image signals are subject to error in both dimensions of space and time. This assumption constrains models of motion perception to be based upon principal component regression rather than linear regression. It is shown that response suppression of model complex cell neurons that input into the model may account for (1) increases in perceived speed after adaptation to static patterns and testing with slowly moving patterns, (2) significant increases in perceived speed after adaptation to patterns moving at a medium speed and testing at high speed, and (3) decreases in perceived speed in the opponent direction to a quickly moving adapting signal. Neither of predictions (2) or (3) are general features of established accounts of motion detection by visual processes based upon linear regression. Comparisons of the proposed model's speed transfer function with existing psychophysical data suggests that the visual system processes motion signals with the tacit assumption that image measurements are subject to error in both space and time. Received: 24 January 2000 / Accepted in revised form: 8 May 2000     

Global Motion Percept Mediated through Integration of Barber Poles Presented in Bilateral Visual Hemifields

How is motion information that has been obtained through multiple viewing apertures integrated to form a global motion percept? We investigated the mechanisms of motion integration across apertures in two hemifields by presenting gratings through two rectangles (that form the dual barber poles) and recording the perceived direction of motion by human observers. To this end, we presented dual barber poles in conditions with various inter-component distances between the apertures and evaluated the degree to which the hemifield information was integrated by measuring the magnitude of the perceived barber pole illusion. Surprisingly, when the inter-component distance between the two apertures was short, the perceived direction of motion of the dual barber poles was similar to that of a single barber pole formed by the concatenation of the two component barber poles, indicating motion integration is achieved through a simple concatenation mechanism. We then presented dual barber poles in which the motion and contour properties of the two component barber poles differed to characterize the constraints underlying cross-hemifield integration. We found that integration is achieved only when phase, speed, wavelength, temporal frequency, and duty cycle are identical in the two barber poles, but can remain robust when the contrast of the two component barber poles differs substantially. We concluded that a motion stimulus presented in bilateral hemifields tends to be integrated to yield a global percept with a substantial tolerance for spatial distance and contrast difference.     

The asynchrony of consciousness.

We present below a simple hypothesis on what we believe is a characteristic of visual consciousness. It is derived from facts about the visual brain revealed in the past quarter of a century, but it relies most especially on psychophysical evidence which shows that different attributes of the visual scene are consciously perceived at different times. This temporal asynchrony in visual perception reveals, we believe, a plurality of visual consciousnesses that are asynchronous with respect to each other, reflecting the modular organization of the visual brain. We further hypothesize that when two attributes (e.g. colour and motion) are presented simultaneously, the activity of cells in a given processing system is sufficient to create a conscious experience of the corresponding attribute (e.g. colour), without the necessity for interaction with the activities of cells in other processing systems (e.g. motion). Thus, any binding of the activity of cells in different systems should be more properly thought of as a binding of the conscious experiences generated in each system.     

Component and intrinsic motion integrate in ‘dancing bar’ illusion

We introduce a new illusion that contradicts common assumptions in the field of visual motion perception. When an unoccluded bar moves at certain speeds and oscillates at certain frequencies, the perceived direction of the bar is not predicted by its intrinsic terminators but is biased to move in the direction orthogonal to its orientation. It appears that the veridical terminator motions are integrated with spurious component motion signals, generating an at times complex pattern of motion around an apparently closed loop path. In the absence of oscillation the effect does not occur. Several factors, including optimal angle, speed, and oscillating distance of the bar, are quantified and possible mechanisms are discussed. In a model, we suggest that the effect arises because of the failure to inhibit spurious component motion signals arising from contours that are nearly oriented along the direction of true motion. Electronic Supplementary Material The online version of this article (doi:) material, which is available to authorized users.