Human visual system integrates color signals along a motion trajectory

Whether fundamental visual attributes, such as color, motion, and shape, are analyzed separately in specialized pathways has been one of the central questions of visual neuroscience. Although recent studies have revealed various forms of cross-attribute interactions, including significant contributions of color signals to motion processing, it is still widely believed that color perception is relatively independent of motion processing. Here, we report a new color illusion, motion-induced color mixing, in which moving bars, the color of each of which alternates between two colors (e.g., red and green), are perceived as the mixed color (e.g., yellow) even though the two colors are never superimposed on the retina. The magnitude of color mixture is significantly stronger than that expected from direction-insensitive spatial integration of color signals. This illusion cannot be ascribed to optical image blurs, including those induced by chromatic aberration, or to involuntary eye movements of the observer. Our findings indicate that color signals are integrated not only at the same retinal location, but also along a motion trajectory. It is possible that this neural mechanism helps us to see veridical colors for moving objects by reducing motion blur, as in the case of luminance-based pattern perception.     

Seeing invisible motion: a human FMRI study

A common view about visual consciousness is that it could arise when and where activity reaches some higher level of processing along the cortical hierarchy. Reports showing that activity in striate cortex can be dissociated from awareness , whereas the latter modulates activity in higher areas , point in this direction. In the specific case of visual motion, a central, "perceptual" role has been assigned to area V5: several human and monkey studies have shown V5 activity to correlate with the motion percept. Here we show that activity in this and other higher cortical areas can be also dissociated from perception and follow the physical stimulus instead. The motion information in a peripheral grating modulated fMRI responses, despite being invisible to human volunteers: under crowding conditions , areas V3A, V5, and parietal cortex still showed increased activity when the grating was moving compared to when it was flickering. We conclude that stimulus-specific activation of higher cortical areas does not necessarily result in awareness of the underlying stimulus.     

Attention to surfaces modulates motion processing in extrastriate area MT

In the visual system, early atomized representations are grouped into higher-level entities through processes of perceptual organization. Here we present neurophysiological evidence that a representation of a simple object, a surface defined by color and motion, can be the unit of attentional selection at an early stage of visual processing. Monkeys were cued by the color of a fixation spot to attend to one of two transparent random-dot surfaces, one red and one green, which occupied the same region of space. Motion of the attended surface drove neurons in the middle temporal (MT) visual area more strongly than physically identical motion of the non-attended surface, even though both occurred within the spotlight of attention. Surface-based effects of attention persisted even without differential surface coloring, but attentional modulation was stronger with color. These results show that attention can select surface representations to modulate visual processing as early as cortical area MT.     

Semantic Wavelet-Induced Frequency-Tagging (SWIFT) Periodically Activates Category Selective Areas While Steadily Activating Early Visual Areas

Primate visual systems process natural images in a hierarchical manner: at the early stage, neurons are tuned to local image features, while neurons in high-level areas are tuned to abstract object categories. Standard models of visual processing assume that the transition of tuning from image features to object categories emerges gradually along the visual hierarchy. Direct tests of such models remain difficult due to confounding alteration in low-level image properties when contrasting distinct object categories. When such contrast is performed in a classic functional localizer method, the desired activation in high-level visual areas is typically accompanied with activation in early visual areas. Here we used a novel image-modulation method called SWIFT (semantic wavelet-induced frequency-tagging), a variant of frequency-tagging techniques. Natural images modulated by SWIFT reveal object semantics periodically while keeping low-level properties constant. Using functional magnetic resonance imaging (fMRI), we indeed found that faces and scenes modulated with SWIFT periodically activated the prototypical category-selective areas while they elicited sustained and constant responses in early visual areas. SWIFT and the localizer were selective and specific to a similar extent in activating category-selective areas. Only SWIFT progressively activated the visual pathway from low- to high-level areas, consistent with predictions from standard hierarchical models. We confirmed these results with criterion-free methods, generalizing the validity of our approach and show that it is possible to dissociate neural activation in early and category-selective areas. Our results provide direct evidence for the hierarchical nature of the representation of visual objects along the visual stream and open up future applications of frequency-tagging methods in fMRI.     

Neural Mechanisms of Cortical Motion Computation Based on a Neuromorphic Sensory System

The visual cortex analyzes motion information along hierarchically arranged visual areas that interact through bidirectional interconnections. This work suggests a bio-inspired visual model focusing on the interactions of the cortical areas in which a new mechanism of feedforward and feedback processing are introduced. The model uses a neuromorphic vision sensor (silicon retina) that simulates the spike-generation functionality of the biological retina. Our model takes into account two main model visual areas, namely V1 and MT, with different feature selectivities. The initial motion is estimated in model area V1 using spatiotemporal filters to locally detect the direction of motion. Here, we adapt the filtering scheme originally suggested by Adelson and Bergen to make it consistent with the spike representation of the DVS. The responses of area V1 are weighted and pooled by area MT cells which are selective to different velocities, i.e. direction and speed. Such feature selectivity is here derived from compositions of activities in the spatio-temporal domain and integrating over larger space-time regions (receptive fields). In order to account for the bidirectional coupling of cortical areas we match properties of the feature selectivity in both areas for feedback processing. For such linkage we integrate the responses over different speeds along a particular preferred direction. Normalization of activities is carried out over the spatial as well as the feature domains to balance the activities of individual neurons in model areas V1 and MT. Our model was tested using different stimuli that moved in different directions. The results reveal that the error margin between the estimated motion and synthetic ground truth is decreased in area MT comparing with the initial estimation of area V1. In addition, the modulated V1 cell activations shows an enhancement of the initial motion estimation that is steered by feedback signals from MT cells.     

Adaptive Colour Contrast Coding in the Salamander Retina Efficiently Matches Natural Scene Statistics

The visual system continually adjusts its sensitivity to the statistical properties of the environment through an adaptation process that starts in the retina. Colour perception and processing is commonly thought to occur mainly in high visual areas, and indeed most evidence for chromatic colour contrast adaptation comes from cortical studies. We show that colour contrast adaptation starts in the retina where ganglion cells adjust their responses to the spectral properties of the environment. We demonstrate that the ganglion cells match their responses to red-blue stimulus combinations according to the relative contrast of each of the input channels by rotating their functional response properties in colour space. Using measurements of the chromatic statistics of natural environments, we show that the retina balances inputs from the two (red and blue) stimulated colour channels, as would be expected from theoretical optimal behaviour. Our results suggest that colour is encoded in the retina based on the efficient processing of spectral information that matches spectral combinations in natural scenes on the colour processing level.     

Smooth Pursuit Eye Movements Improve Temporal Resolution for Color Perception

Human observers see a single mixed color (yellow) when different colors (red and green) rapidly alternate. Accumulating evidence suggests that the critical temporal frequency beyond which chromatic fusion occurs does not simply reflect the temporal limit of peripheral encoding. However, it remains poorly understood how the central processing controls the fusion frequency. Here we show that the fusion frequency can be elevated by extra-retinal signals during smooth pursuit. This eye movement can keep the image of a moving target in the fovea, but it also introduces a backward retinal sweep of the stationary background pattern. We found that the fusion frequency was higher when retinal color changes were generated by pursuit-induced background motions than when the same retinal color changes were generated by object motions during eye fixation. This temporal improvement cannot be ascribed to a general increase in contrast gain of specific neural mechanisms during pursuit, since the improvement was not observed with a pattern flickering without changing position on the retina or with a pattern moving in the direction opposite to the background motion during pursuit. Our findings indicate that chromatic fusion is controlled by a cortical mechanism that suppresses motion blur. A plausible mechanism is that eye-movement signals change spatiotemporal trajectories along which color signals are integrated so as to reduce chromatic integration at the same locations (i.e., along stationary trajectories) on the retina that normally causes retinal blur during fixation.     

Lower visual field advantage for motion segmentation during high competition for selection

A series of visual enumeration tasks were conducted investigating the role of the dorsal visual stream in motion segmentation. Cortical areas representing the lower visual field have greater connections with the parietal cortex and should therefore show an advantage for processes driven by the dorsal stream (Previc, 1990). We looked for differences in processing displays in the upper versus lower visual field when targets required segmentation from distractors in an enumeration task. In a baseline condition, random configurations of moving and static items were presented briefly (200 ms) to the upper or lower visual field. Fast and efficient enumeration took place both for moving targets and for static targets presented alone; there was no effect of visual field. In contrast, for moving targets, a lower visual field advantage was found when the inclusion of static distractors demanded segmentation by motion. This disappeared at the smaller display sizes when the targets were presented in canonical patterns. The results are consistent with segmentation of moving targets from static distractors being mediated by dorsal regions of the visual cortex, particularly under conditions of high load (non-canonical patterns). These regions show greater sensitivity to the lower visual field and to magnocellular-based input.     

Interstitial cells of Cajal in the striated musculature of the mouse esophagus

A key feature of signal processing in the mammalian retina is parallel processing, where the segregation of visual information, e.g., brightness, darkness, and color, starts at the first synapse in the retina, the photoreceptor synapse. These various aspects are transmitted in parallel from the input neurons of the retina, the photoreceptor cells, through the interconnecting bipolar cells, to the output neurons, the ganglion cells. The photoreceptors and bipolar cells release a single excitatory neurotransmitter, glutamate, at their synapses. This parsimony is contrasted by the expression of a plethora of glutamate receptors, receptor subunits, and isoforms. The detailed knowledge of the synaptic distribution of glutamate receptors thus is of major importance in understanding the mechanisms of retinal signal processing. This review intends to highlight recent studies on the distribution of glutamate receptors at the photoreceptor synapses of the mammalian retina.     

Posterior parietal cortex neurons encode target motion in world-centered coordinates

The motion areas of posterior parietal cortex extract information on visual motion for perception as well as for the guidance of movement. It is usually assumed that neurons in posterior parietal cortex represent visual motion relative to the retina. Current models describing action guided by moving objects work successfully based on this assumption. However, here we show that the pursuit-related responses of a distinct group of neurons in area MST of monkeys are at odds with this view. Rather than signaling object image motion on the retina, they represent object motion in world-centered coordinates. This representation may simplify the coordination of object-directed action and ego motion-invariant visual perception.     

Anteroposterior dynamic balance reactions induced by circular translation of the visual field

The anteroposterior sway of subjects under conditions of spontaneous dynamic balance on a wobbly platform was measured during visual stimulation by a visual target executing a circular trajectory in the frontal plane. The target was either a component of the whole moving visual scene or moving on a stationary background. With the former stimulation, obtained through the use of rotating prismatic glasses, every point of the visual field appeared to describe a circular trajectory around its real position so that the whole visual field apeared to be circularly translated, undistorted, inducing a binocular pursuit movement. Under these conditions, stereotyped anteroposterior dynamic balance reactions synchronous with the position of the stimulus were elicited. The latter stimulation consisted of pursuing a luminous target describing a trajectory similar to that of the fixation point seen through the rotating prisms on the same, this time stable, visual background. Although pursuit eye movements were comparable, as demonstrated by electro-oculographic recordings, no stereotyped equilibration reaction was induced. It is concluded that the translatory motion of the background image on the retina in the latter experiments contributed to the body's stability as well as to the perception of a stable environment.     

Form and motion coherence activate independent, but not dorsal/ventral segregated, networks in the human brain

There is much evidence in primates' visual processing for distinct mechanisms involved in object recognition and encoding object position and motion, which have been identified with 'ventral' and 'dorsal' streams, respectively, of the extra-striate visual areas [1] [2] [3]. This distinction may yield insights into normal human perception, its development and pathology. Motion coherence sensitivity has been taken as a test of global processing in the dorsal stream [4] [5]. We have proposed an analogous 'form coherence' measure of global processing in the ventral stream [6]. In a functional magnetic resonance imaging (fMRI) experiment, we found that the cortical regions activated by form coherence did not overlap with those activated by motion coherence in the same individuals. Areas differentially activated by form coherence included regions in the middle occipital gyrus, the ventral occipital surface, the intraparietal sulcus, and the temporal lobe. Motion coherence activated areas consistent with those previously identified as V5 and V3a, the ventral occipital surface, the intraparietal sulcus, and temporal structures. Neither form nor motion coherence activated area V1 differentially. Form and motion foci in occipital, parietal, and temporal areas were nearby but showed almost no overlap. These results support the idea that form and motion coherence test distinct functional brain systems, but that these do not necessarily correspond to a gross anatomical separation of dorsal and ventral processing streams.     

Using low-level filters to encode spatial displacements of visual stimuli

Directional responses to visual stimuli were analysed with the aid of a minimal computational model. The model is based upon arrays of motion sensors whose receptive fields are modified versions of those (difference-of-Gaussians) used to describe mechanisms in popular spatial vision models. In the model antagonistic influences on each motion sensor were assumed to: (1) arise from spatially non-aligned areas of the retina; and (2) to follow different time courses. Implications of the model were explored with simulations, and parallel psychophysical data were collected. Visual behaviours chosen for relatively detailed analysis were judgments of the temporal order of onset of two spatially displaced stimuli and motion aftereffects generated with discontinuously moving, sine-wave gratings.     

Sensory response patterns and the evolution of visual signal design in anoline lizards

 The evolutionary relationship between visual system response and visual signal design was investigated in four species of anoline lizards which occupy distinctly different habitats. Anoles display with motion patterns of a colorful throat fan called the dewlap. We assessed signal visibility by recording evoked potentials from the optic tectum in response to a moving stimulus flag (a dewlap-like stimulus), and, in one species, by testing behavioral response. The motion pattern, intensity and spectral quality of the stimulus flag, and the background against which it was viewed, were independently manipulated. In all cases, high-velocity motion patterns with a high percentage of brightness contrast between stimulus and background produced the greatest response. Differences in spectral quality between stimulus and background (color contrast) had no effect on tectal responses, but did influence the behaviorally measured detection probability. Using habitat light data we estimated the visibility of the dewlap of each species in different natural habitats. Each species' dewlap was highly visible in its own habitat, but some were much less visible in the habitats of some other species. Habitat light conditions and visual system response properties appear to have constrained the evolution of dewlap design, in at least some of the species. Accepted: May 1998     

CORRELATIONAL SELECTION FOR COLOR PATTERN AND ANTIPREDATOR BEHAVIOR IN THE GARTER SNAKE THAMNOPHIS ORDINOIDES

Correlational selection favors combinations of traits and is a key element of many models of phenotypic and genetic evolution. Multiple regression techniques for measuring selection allow for the direct estimation of correlational selection gradients, yet few studies in natural populations have investigated this process. Color patterns and antipredator behaviors of snakes are thought to function interactively in predator escape and therefore may be subject to correlational selection. To investigate this hypothesis, I studied the survivorship of juvenile garter snakes, Thamnophis ordinoides, as a function of a suite of escape behaviors and color pattern. The only natural selection detected favored opposite combinations of stripedness of the color pattern and the tendency to perform during escape evasive behaviors called reversals. This selection presumably results from optical illusions created by moving patterns and their effects on visually foraging predators. Analysis of the bivariate selection surface shows that pure correlational selection can be thought of as a series of linear selection functions on one trait whose slopes depend on the value of the second trait. Alternatively, viewing the selection surface along its major axes reveals stabilizing and disruptive components of correlational selection. It is further shown that correlational selection alone can promote genetic variance and covariance within a generation. This phenomenon may be partially responsible for the extreme variation in color pattern and the genetic covariance between color pattern and behavior observed in natural populations of T. ordinoides.     

Various background pattern-effect on saccadic suppression.

It has been proved that the saccadic suppression is a phenomenon closely related to the presence of contours and structures in the visual field. Experiments were performed to clarify whether the structured background influences the pattern of attention distribution (making the stimulus detection more difficult) or whether the elevation of visual threshold is due to the "masking' effect of the moving background image over the retina. Two types of backgrounds were used therefore: those with symbolic meaning in the processing of which "psychological' mechanisms are presumably involved like picture reproductions of famous painters and photographs of nudes, and those lacking semantic significance like computer figures composed of randomly distributed black and white squares with different grain expressed as the entropy of the pattern. The results show that saccadic suppression is primarily a consequence of peripheral mechanisms, probably of lateral inhibition in the visual field occurring in the presence of moving edges over the retina. Psychological factors have to be excluded as being fundamental for saccadic suppression.     

Characteristics of Motor Resonance Predict the Pattern of Flash-Lag Effects for Biological Motion

Background

When a moving stimulus and a briefly flashed static stimulus are physically aligned in space the static stimulus is perceived as lagging behind the moving stimulus. This vastly replicated phenomenon is known as the Flash-Lag Effect (FLE). For the first time we employed biological motion as the moving stimulus, which is important for two reasons. Firstly, biological motion is processed by visual as well as somatosensory brain areas, which makes it a prime candidate for elucidating the interplay between the two systems with respect to the FLE. Secondly, discussions about the mechanisms of the FLE tend to recur to evolutionary arguments, while most studies employ highly artificial stimuli with constant velocities.

Methodology/Principal Finding

Since biological motion is ecologically valid it follows complex patterns with changing velocity. We therefore compared biological to symbolic motion with the same acceleration profile. Our results with 16 observers revealed a qualitatively different pattern for biological compared to symbolic motion and this pattern was predicted by the characteristics of motor resonance: The amount of anticipatory processing of perceived actions based on the induced perspective and agency modulated the FLE.

Conclusions/Significance

Our study provides first evidence for an FLE with non-linear motion in general and with biological motion in particular. Our results suggest that predictive coding within the sensorimotor system alone cannot explain the FLE. Our findings are compatible with visual prediction (Nijhawan, 2008) which assumes that extrapolated motion representations within the visual system generate the FLE. These representations are modulated by sudden visual input (e.g. offset signals) or by input from other systems (e.g. sensorimotor) that can boost or attenuate overshooting representations in accordance with biased neural competition (Desimone & Duncan, 1995).     

Robust velocity computation from a biologically motivated model of motion perception

Current computational models of motion processing in the primate motion pathway do not cope well with image sequences in which a moving pattern is superimposed upon a static texture. The use of non-linear operations and the need for contrast normalization in motion models mean that the separation of the influences of moving and static patterns on the motion computation is not trivial. Therefore, the response to the superposition of static and moving patterns provides an important means of testing various computational strategies. Here we describe a computational model of motion processing in the visual cortex, one of the advantages of which is that it is highly resistant to interference from static patterns.     

Implicit attentional selection of bound visual features

Traditionally, research on visual attention has been focused on the processes involved in conscious, explicit selection of task-relevant sensory input. Recently, however, it has been shown that attending to a specific feature of an object automatically increases neural sensitivity to this feature throughout the visual field. Here we show that directing attention to a specific color of an object results in attentional modulation of the processing of task-irrelevant and not consciously perceived motion signals that are spatiotemporally associated with this color throughout the visual field. Such implicit cross-feature spreading of attention takes place according to the veridical physical associations between the color and motion signals, even under special circumstances when they are perceptually misbound. These results imply that the units of implicit attentional selection are spatiotemporally colocalized feature clusters that are automatically bound throughout the visual field.     

Dynamic shape integration in extrastriate cortex

BACKGROUND: In anorthoscopic viewing conditions, observers can perceive a moving object through a narrow slit even when only portions of its contour are visible at any time. We used fMRI to examine the contribution of early and later visual cortical areas to dynamic shape integration. Observers' success at integrating the shape of the slit-viewed object was manipulated by varying the degree to which the stimulus was dynamically distorted. Line drawings of common objects were either moderately distorted, strongly distorted, or shown undistorted. Phenomenologically, increasing the stimulus distortion made both object shape and motion more difficult to perceive.RESULTS: We found that bilateral cortical activity in portions of the ventral occipital cortex, corresponding to known object areas within the lateral occipital complex (LOC), was inversely correlated with the degree of stimulus distortion. We found that activity in left MT+, the human cortical area specialized for motion, showed a similar pattern as the ventral occipital region. The LOC also showed greater activity to a fully visible moving object than to the undistorted slit-viewed object. Area MT+, however, showed more equivalent activity to both the slit-viewed and fully visible moving objects.CONCLUSIONS: In early retinotopic cortex, the distorted and undistorted stimuli elicited the same amount of activity. Higher visual areas, however, were correlated with the percept of the coherent object, and this correlation suggests that the shape integration is mediated by later visual cortical areas. Motion information from the dorsal stream may project to the LOC to produce the shape percept.