The visual processing of motion-defined transparency

Our understanding of how the visual system processes motion transparency, the phenomenon by which multiple directions of motion are perceived to coexist in the same spatial region, has grown considerably in the past decade. There is compelling evidence that the process is driven by global-motion mechanisms. Consequently, although transparently moving surfaces are readily segmented over an extended space, the visual system cannot separate two motion signals that coexist in the same local region. A related issue is whether the visual system can detect transparently moving surfaces simultaneously or whether the component signals encounter a serial 'bottleneck' during their processing. Our initial results show that, at sufficiently short stimulus durations, observers cannot accurately detect two superimposed directions; yet they have no difficulty in detecting one pattern direction in noise, supporting the serial-bottleneck scenario. However, in a second experiment, the difference in performance between the two tasks disappears when the component patterns are segregated. This discrepancy between the processing of transparent and non-overlapping patterns may be a consequence of suppressed activity of global-motion mechanisms when the transparent surfaces are presented in the same depth plane. To test this explanation, we repeated our initial experiment while separating the motion components in depth. The marked improvement in performance leads us to conclude that transparent motion signals are represented simultaneously.     

Direct evidence for encoding of motion streaks in human visual cortex

Temporal integration in the visual system causes fast-moving objects to generate static, oriented traces (‘motion streaks’), which could be used to help judge direction of motion. While human psychophysics and single-unit studies in non-human primates are consistent with this hypothesis, direct neural evidence from the human cortex is still lacking. First, we provide psychophysical evidence that faster and slower motions are processed by distinct neural mechanisms: faster motion raised human perceptual thresholds for static orientations parallel to the direction of motion, whereas slower motion raised thresholds for orthogonal orientations. We then used functional magnetic resonance imaging to measure brain activity while human observers viewed either fast (‘streaky’) or slow random dot stimuli moving in different directions, or corresponding static-oriented stimuli. We found that local spatial patterns of brain activity in early retinotopic visual cortex reliably distinguished between static orientations. Critically, a multivariate pattern classifier trained on brain activity evoked by these static stimuli could then successfully distinguish the direction of fast (‘streaky’) but not slow motion. Thus, signals encoding static-oriented streak information are present in human early visual cortex when viewing fast motion. These experiments show that motion streaks are present in the human visual system for faster motion.     

Integration of Motion Responses Underlying Directional Motion Anisotropy in Human Early Visual Cortical Areas

Recent imaging studies have reported directional motion biases in human visual cortex when perceiving moving random dot patterns. It has been hypothesized that these biases occur as a result of the integration of motion detector activation along the path of motion in visual cortex. In this study we investigate the nature of such motion integration with functional MRI (fMRI) using different motion stimuli. Three types of moving random dot stimuli were presented, showing either coherent motion, motion with spatial decorrelations or motion with temporal decorrelations. The results from the coherent motion stimulus reproduced the centripetal and centrifugal directional motion biases in V1, V2 and V3 as previously reported. The temporally decorrelated motion stimulus resulted in both centripetal and centrifugal biases similar to coherent motion. In contrast, the spatially decorrelated motion stimulus resulted in small directional motion biases that were only present in parts of visual cortex coding for higher eccentricities of the visual field. In combination with previous results, these findings indicate that biased motion responses in early visual cortical areas most likely depend on the spatial integration of a simultaneously activated motion detector chain.     

Computational modelling of motion detectors: responses to two-frame displays

Schemes for motion detection fall into two classes. Reichardt correlators compare spatial luminance patterns at two locations at different times; gradient detectors compare spatial and temporal luminance gradients. Both are candidate operators for biological and machine vision systems. A large body of perceptual data exists, defining the properties of motion detectors used by human observers, which can form a basis for determining which class of detector is appropriate for the human visual system. Plausible versions of each detector were implemented, and their responses to a variety of two-frame stimuli were computed. Results indicated that both detectors can predict most of the data, but on balance gradient detectors offer the best working hypothesis for motion detection by human observers. This conclusion is necessarily limited to the type of stimuli used, and may require modification in the light of responses to continuously moving stimuli.     

Testing visual sensitivity to the speed and direction of motion in lizards

Testing visual sensitivity in any species provides basic information regarding behaviour, evolution, and ecology. However, testing specific features of the visual system provide more empirical evidence for functional applications. Investigation into the sensory system provides information about the sensory capacity, learning and memory ability, and establishes known baseline behaviour in which to gauge deviations (Burghardt, 1977). However, unlike mammalian or avian systems, testing for learning and memory in a reptile species is difficult. Furthermore, using an operant paradigm as a psychophysical measure of sensory ability is likewise as difficult. Historically, reptilian species have responded poorly to conditioning trials because of issues related to motivation, physiology, metabolism, and basic biological characteristics. Here, I demonstrate an operant paradigm used a novel model lizard species, the Jacky dragon (Amphibolurus muricatus) and describe how to test peripheral sensitivity to salient speed and motion characteristics. This method uses an innovative approach to assessing learning and sensory capacity in lizards. I employ the use of random-dot kinematograms (RDKs) to measure sensitivity to speed, and manipulate the level of signal strength by changing the proportion of dots moving in a coherent direction. RDKs do not represent a biologically meaningful stimulus, engages the visual system, and is a classic psychophysical tool used to measure sensitivity in humans and other animals. Here, RDKs are displayed to lizards using three video playback systems. Lizards are to select the direction (left or right) in which they perceive dots to be moving. Selection of the appropriate direction is reinforced by biologically important prey stimuli, simulated by computer-animated invertebrates.     

Long-range interactions in the spatial integration of motion signals.

When a sinewave grating is moving within a cross-shaped aperture, a strongly multi-stable phenomenon is perceived. The percept switches between the coherence of an extended surface moving in a single direction and the segregation of two patterned strips sliding across each other in directions parallel to the branches of the cross. We studied how the balance between these two percepts is affected by the length of the arms and by the shape of their ends. We report here that human observers report the segregation into two surfaces more often when the branches of the cross are extended, and when the small sides of the arms are oriented parallel to the grating. Two kinds of early motion signals interact in the crossed barber-pole stimulus: (a) the signals extracted in the middle of the bars are ambiguous with regard to their direction, and usually would be interpreted as motion normal to the grating orientation; (b) the signals from regions where the grating is intersected by the borders of the aperture convey motion signals in direction of the border. Our results show that the global appearance of our display can be dramatically influenced by the reliability of motion signals located in small regions that may be separated by large distances. To explain this long-range effect, we tentatively propose the existence of a representation level situated between the extraction of low-level local signals and the final global percept. The postulated processing level is concerned with the segmenting of the entire image into surfaces that are likely to belong to the same object, even if they are not contiguous in space. This hypothetical mechanism involves the construction of coarse-scale 'patches' from the local motion signal distributions, each carrying a single velocity associated with a certain degree of reliability. Our experiments indicate that the probability of grouping together similar patches depends on their respective reliabilities.     

Weighting Mean and Variability during Confidence Judgments

Humans can not only perform some visual tasks with great precision, they can also judge how good they are in these tasks. However, it remains unclear how observers produce such metacognitive evaluations, and how these evaluations might be dissociated from the performance in the visual task. Here, we hypothesized that some stimulus variables could affect confidence judgments above and beyond their impact on performance. In a motion categorization task on moving dots, we manipulated the mean and the variance of the motion directions, to obtain a low-mean low-variance condition and a high-mean high-variance condition with matched performances. Critically, in terms of confidence, observers were not indifferent between these two conditions. Observers exhibited marked preferences, which were heterogeneous across individuals, but stable within each observer when assessed one week later. Thus, confidence and performance are dissociable and observers’ confidence judgments put different weights on the stimulus variables that limit performance.     

Looking at Op Art from a computational viewpoint

Arts history tells an exciting story about repeated attempts to represent features that are crucial for the understanding of our environment and which, at the same time, go beyond the inherently two-dimensional nature of a flat painting surface: depth and motion. In the twentieth century, Op artists such as Bridget Riley began to experiment with simple black and white patterns that do not represent motion in an artistic way but actually create vivid dynamic illusions in static pictures. The cause of motion illusions in such paintings is still a matter of debate. The role of involuntary eye movements in this phenomenon is studied here with a computational approach. The possible consequences of shifting the retinal image of synthetic wave gratings, dubbed as 'riloids', were analysed by a two-dimensional array of motion detectors (2DMD model), which generates response maps representing the spatial distribution of motion signals generated by such a stimulus. For a two-frame sequence reflecting a saccadic displacement, these motion signal maps contain extended patches in which local directions change only little. These directions, however, do not usually precisely correspond to the direction of pattern displacement that can be expected from the geometry of the curved gratings as an instance of the so-called 'aperture problem'. The patchy structure of the simulated motion detector response to the displacement of riloids resembles the motion illusion, which is not perceived as a coherent shift of the whole pattern but as a wobbling and jazzing of ill-defined regions. Although other explanations are not excluded, this might support the view that the puzzle of Op Art motion illusions could potentially have an almost trivial solution in terms of small involuntary eye movement leading to image shifts that are picked up by well-known motion detectors in the early visual system. This view can have further consequences for our understanding of how the human visual system usually compensates for eye movements, in order to let us perceive a stable world despite continuous image shifts generated by gaze instability.     

Motion-based analysis of spatial patterns by the human visual system

BACKGROUND: It is known that the visibility of patterns presented through stationary multiple slits is significantly improved by pattern movements. This study investigated whether this spatiotemporal pattern interpolation is supported by motion mechanisms, as opposed to the general belief that the human visual cortex initially analyses spatial patterns independent of their movements. RESULTS: Psychophysical experiments showed that multislit viewing could not be ascribed to such motion-irrelevant factors as retinal painting by tracking eye movements or an increase in the number of views by pattern movements. Pattern perception was more strongly impaired by the masking noise moving in the same direction than by the noise moving in the opposite direction, which indicates the direction selectivity of the pattern interpolation mechanism. A direction-selective impairment of pattern perception by motion adaptation also indicates the direction selectivity of the interpolation mechanism. Finally, the map of effective spatial frequencies, estimated by a reverse-correlation technique, indicates observers' perception of higher spatial frequencies, the recovery of which is theoretically impossible without the aid of motion information. CONCLUSIONS: These results provide clear evidence against the notion of separate analysis of pattern and motion. The visual system uses motion mechanisms to integrate spatial pattern information along the trajectory of pattern movement in order to obtain clear perception of moving patterns. The pattern integration mechanism is likely to be direction-selective filtering by V1 simple cells, but the integration of the local pattern information into a global figure should be guided by a higher-order motion mechanism such as MT pattern cells.     

Propagation of photon noise and information transfer in visual motion detection

The extraction of the direction of motion from the time varying retinal images is one of the most basic tasks any visual system is confronted with. However, retinal images are severely corrupted by photon noise, in particular at low light levels, thus limiting the performance of motion detection mechanisms of what sort so ever. Here, we study how photon noise propagates through an array of Reichardt-type motion detectors that are commonly believed to underlie fly motion vision. We provide closed-form analytical expressions of the signal and noise spectra at the output of such a motion detector array. We find that Reichardt detectors reveal favorable noise suppression in the frequency range where most of the signal power resides. Most notably, due to inherent adaptive properties, the transmitted information about stimulus velocity remains nearly constant over a large range of velocity entropies. Action editor: Matthew Wiener     

Slow and fast visual motion channels have independent binocular-rivalry stages

We have previously reported a transparent motion after-effect indicating that the human visual system comprises separate slow and fast motion channels. Here, we report that the presentation of a fast motion in one eye and a slow motion in the other eye does not result in binocular rivalry but in a clear percept of transparent motion. We call this new visual phenomenon 'dichoptic motion transparency' (DMT). So far only the DMT phenomenon and the two motion after-effects (the 'classical' motion after-effect, seen after motion adaptation on a static test pattern, and the dynamic motion after-effect, seen on a dynamic-noise test pattern) appear to isolate the channels completely. The speed ranges of the slow and fast channels overlap strongly and are observer dependent. A model is presented that links after-effect durations of an observer to the probability of rivalry or DMT as a function of dichoptic velocity combinations. Model results support the assumption of two highly independent channels showing only within-channel rivalry, and no rivalry or after-effect interactions between the channels. The finding of two independent motion vision channels, each with a separate rivalry stage and a private line to conscious perception, might be helpful in visualizing or analysing pathways to consciousness.     

Discrimination of random dot texture patterns in bluegill sunfish,Lepomis macrochirus

In order to study the ability of fish to perceive and distinguish textures visually, bluegill sunfish (Lepomis macrochirus) were trained to discriminate between pairs of artificial texture patterns. Random dot patterns with different statistical dot distributions were presented to the fish as artificial texture patterns. The results indicate that bluegills have the ability to discriminate many pairs of patterns with different statistical features of dot distributions which have different appearance in texture. This suggests that texture could be one of the important visual features bluegill sunfish recognize and utilize.     

Shape Invariant Coding of Motion Direction in Somatosensory Cortex

Invariant representations of stimulus features are thought to play an important role in producing stable percepts of objects. In the present study, we assess the invariance of neural representations of tactile motion direction with respect to other stimulus properties. To this end, we record the responses evoked in individual neurons in somatosensory cortex of primates, including areas 3b, 1, and 2, by three types of motion stimuli, namely scanned bars and dot patterns, and random dot displays, presented to the fingertips of macaque monkeys. We identify a population of neurons in area 1 that is highly sensitive to the direction of stimulus motion and whose motion signals are invariant across stimulus types and conditions. The motion signals conveyed by individual neurons in area 1 can account for the ability of human observers to discriminate the direction of motion of these stimuli, as measured in paired psychophysical experiments. We conclude that area 1 contains a robust representation of motion and discuss similarities in the neural mechanisms of visual and tactile motion processing.     

Context-dependent changes in functional circuitry in visual area MT

Animals can flexibly change their behavior in response to a particular sensory stimulus; the mapping between sensory and motor representations in the brain must therefore be flexible as well. Changes in the correlated firing of pairs of neurons may provide a metric of changes in functional circuitry during behavior. We studied dynamic changes in functional circuitry by analyzing the noise correlations of simultaneously recorded MT neurons in two behavioral contexts: one that promotes cooperative interactions between the two neurons and another that promotes competitive interactions. We found that identical visual stimuli give rise to differences in noise correlation in the two contexts, suggesting that MT neurons receive inputs of central origin whose strength changes with the task structure. The data are consistent with a mixed feature-based attentional strategy model in which the animal sometimes alternates attention between opposite directions of motion and sometimes attends to the two directions simultaneously.     

A multilayer neural network model for perception of rotational motion

A multilayer neural nerwork model for the perception of rotational motion has been developed usingReichardt's motion detector array of correlation type, Kohonen's self-organized feature map and Schuster-Wagner's oscillating neural network. It is shown that the unsupervised learning could make the neurons on the second layer of the network tend to be self-organized in a form resembling columnar organization of selective directions in area MT of the primate's visual cortex. The output layer can interpret rotation information and give the directions and velocities of rotational motion. The computer simulation results are in agreement with some psychophysical observations of rotation-al perception. It is demonstrated that the temporal correlation between the oscillating neurons would be powerful for solving the "binding problem" of shear components of rotational motion.     

Feature-based attentional modulations in the absence of direct visual stimulation

When faced with a crowded visual scene, observers must selectively attend to behaviorally relevant objects to avoid sensory overload. Often this selection process is guided by prior knowledge of a target-defining feature (e.g., the color red when looking for an apple), which enhances the firing rate of visual neurons that are selective for the attended feature. Here, we used functional magnetic resonance imaging and a pattern classification algorithm to predict the attentional state of human observers as they monitored a visual feature (one of two directions of motion). We find that feature-specific attention effects spread across the visual field-even to regions of the scene that do not contain a stimulus. This spread of feature-based attention to empty regions of space may facilitate the perception of behaviorally relevant stimuli by increasing sensitivity to attended features at all locations in the visual field.     

The evolution of alternative adaptive strategies for effective communication in noisy environments

Animals communicating socially are expected to produce signals that are conspicuous within the habitats in which they live. The particular way in which a species adapts to its environment will depend on its ancestral condition and evolutionary history. At this point, it is unclear how properties of the environment and historical factors interact to shape communication. Tropical Anolis lizards advertise territorial ownership using visual displays in habitats where visual motion or "noise" from windblown vegetation poses an acute problem for the detection of display movements. We studied eight Anolis species that live in similar noise environments but belong to separate island radiations with divergent evolutionary histories. We found that species on Puerto Rico displayed at times when their signals were more likely to be detected by neighboring males and females (during periods of low noise). In contrast, species on Jamaica displayed irrespective of the level of environmental motion, apparently because these species have a display that is effective in a range of viewing conditions. Our findings appear to reflect a case of species originating from different evolutionary starting points evolving different signal strategies for effective communication in noisy environments.     

Perception of Social Interactions for Spatially Scrambled Biological Motion

It is vitally important for humans to detect living creatures in the environment and to analyze their behavior to facilitate action understanding and high-level social inference. The current study employed naturalistic point-light animations to examine the ability of human observers to spontaneously identify and discriminate socially interactive behaviors between two human agents. Specifically, we investigated the importance of global body form, intrinsic joint movements, extrinsic whole-body movements, and critically, the congruency between intrinsic and extrinsic motions. Motion congruency is hypothesized to be particularly important because of the constraint it imposes on naturalistic action due to the inherent causal relationship between limb movements and whole body motion. Using a free response paradigm in Experiment 1, we discovered that many naïve observers (55%) spontaneously attributed animate and/or social traits to spatially-scrambled displays of interpersonal interaction. Total stimulus motion energy was strongly correlated with the likelihood that an observer would attribute animate/social traits, as opposed to physical/mechanical traits, to the scrambled dot stimuli. In Experiment 2, we found that participants could identify interactions between spatially-scrambled displays of human dance as long as congruency was maintained between intrinsic/extrinsic movements. Violating the motion congruency constraint resulted in chance discrimination performance for the spatially-scrambled displays. Finally, Experiment 3 showed that scrambled point-light dancing animations violating this constraint were also rated as significantly less interactive than animations with congruent intrinsic/extrinsic motion. These results demonstrate the importance of intrinsic/extrinsic motion congruency for biological motion analysis, and support a theoretical framework in which early visual filters help to detect animate agents in the environment based on several fundamental constraints. Only after satisfying these basic constraints could stimuli be evaluated for high-level social content. In this way, we posit that perceptual animacy may serve as a gateway to higher-level processes that support action understanding and social inference.     

Neural population representation hypothesis of visual flow and its illusory after effect in the brain: psychophysics,neurophysiology and computational approaches

The neural representation of motion aftereffects induced by various visual flows (translational, rotational, motion-in-depth, and translational transparent flows) was studied under the hypothesis that the imbalances in discharge activities would occur in favor in the direction opposite to the adapting stimulation in the monkey MST cells (cells in the medial superior temporal area) which can discriminate the mode (i.e., translational, rotational, or motion-in-depth) of the given flow. In single-unit recording experiments conducted on anaesthetized monkeys, we found that the rate of spontaneous discharge and the sensitivity to a test stimulus moving in the preferred direction decreased after receiving an adapting stimulation moving in the preferred direction, whereas they increased after receiving an adapting stimulation moving in the null direction. To consistently explain the bidirectional perception of a transparent visual flow and its unidirectional motion aftereffect by the same hypothesis, we need to assume the existence of two subtypes of MST D cells which show directionally selective responses to a translational flow: component cells and integration cells. Our physiological investigation revealed that the MST D cells could be divided into two types: one responded to a transparent flow by two peaks at the instances when the direction of one of the component flow matched the preferred direction of the cell, and the other responded by a single peak at the instance when the direction of the integrated motion matched the preferred direction. In psychophysical experiments on human subjects, we found evidence for the existence of component and integration representations in the human brain. To explain the different motion perceptions, i.e., two transparent flows during presentation of the flows and a single flow in the opposite direction to the integrated flows after stopping the flow stimuli, we suggest that the pattern-discrimination system can select the motion representation that is consistent with the perception of the pattern from two motion representations. We discuss the computational aspects related to the integration of component motion fields.     

Influence of Correspondence Noise and Spatial Scaling on the Upper Limit for Spatial Displacement in Fully-Coherent Random-Dot Kinematogram Stimuli

Correspondence noise is a major factor limiting direction discrimination performance in random-dot kinematograms [1]. In the current study we investigated the influence of correspondence noise on Dmax, which is the upper limit for the spatial displacement of the dots for which coherent motion is still perceived. Human direction discrimination performance was measured, using 2-frame kinematograms having leftward/rightward motion, over a 200-fold range of dot-densities and a four-fold range of dot displacements. From this data Dmax was estimated for the different dot densities tested. A model was proposed to evaluate the correspondence noise in the stimulus. This model summed the outputs of a set of elementary Reichardt-type local detectors that had receptive fields tiling the stimulus and were tuned to the two directions of motion in the stimulus. A key assumption of the model was that the local detectors would have the radius of their catchment areas scaled with the displacement that they were tuned to detect; the scaling factor k linking the radius to the displacement was the only free parameter in the model and a single value of k was used to fit all of the psychophysical data collected. This minimal, correspondence-noise based model was able to account for 91% of the variability in the human performance across all of the conditions tested. The results highlight the importance of correspondence noise in constraining the largest displacement that can be detected.