Decoding seen and attended motion directions from activity in the human visual cortex

Functional neuroimaging has successfully identified brain areas that show greater responses to visual motion and adapted responses to repeated motion directions. However, such methods have been thought to lack the sensitivity and spatial resolution to isolate direction-selective responses to individual motion stimuli. Here, we used functional magnetic resonance imaging (fMRI) and pattern classification methods to show that ensemble activity patterns in human visual cortex contain robust direction-selective information, from which it is possible to decode seen and attended motion directions. Ensemble activity in areas V1-V4 and MT+/V5 allowed us to decode which of eight possible motion directions the subject was viewing on individual stimulus blocks. Moreover, ensemble activity evoked by single motion directions could effectively predict which of two overlapping motion directions was the focus of the subject's attention and presumably dominant in perception. Our results indicate that feature-based attention can bias direction-selective population activity in multiple visual areas, including MT+/V5 and early visual areas (V1-V4), consistent with gain-modulation models of feature-based attention and theories of early attentional selection. Our approach for measuring ensemble direction selectivity may provide new opportunities to investigate relationships between attentional selection, conscious perception, and direction-selective responses in the human brain.     

Human visual system integrates color signals along a motion trajectory

Whether fundamental visual attributes, such as color, motion, and shape, are analyzed separately in specialized pathways has been one of the central questions of visual neuroscience. Although recent studies have revealed various forms of cross-attribute interactions, including significant contributions of color signals to motion processing, it is still widely believed that color perception is relatively independent of motion processing. Here, we report a new color illusion, motion-induced color mixing, in which moving bars, the color of each of which alternates between two colors (e.g., red and green), are perceived as the mixed color (e.g., yellow) even though the two colors are never superimposed on the retina. The magnitude of color mixture is significantly stronger than that expected from direction-insensitive spatial integration of color signals. This illusion cannot be ascribed to optical image blurs, including those induced by chromatic aberration, or to involuntary eye movements of the observer. Our findings indicate that color signals are integrated not only at the same retinal location, but also along a motion trajectory. It is possible that this neural mechanism helps us to see veridical colors for moving objects by reducing motion blur, as in the case of luminance-based pattern perception.     

A system of insect neurons sensitive to horizontal and vertical image motion connects the medulla and midbrain

Summary The response properties and gross morphologies of neurons that connect the medulla and midbrain in the butterfly Papilio aegeus are described. The neurons presented give direction-selective responses, i.e. they are excited by motion in the preferred direction and the background activity of the cells is inhibited by motion in the opposite, null, direction. The neurons are either maximally sensitive to horizontal motion or to slightly off-axis vertical upward or vertical downward motion, when tested in the frontal visual field. The responses of the cells are dependent on the contrast frequency of the stimulus with peak values at 5–10 Hz. The receptive fields of the medulla neurons are large and are most sensitive in the frontal visual field. Examination of the local and global properties of the receptive fields of the medulla neurons indicates that (1) they are fed by local elementary motion-detectors consistent with the correlation model and (2) there is a non-linear spatial integration mechanism in operation.     

A model for direction selectivity in threshold motion perception

Thresholds were measured for a moving line superimposed on moving sinusoidal gratings. When line and grating moved in the same direction significant subthreshold summation was observed over a range of spatial frequencies. For motion of the line and grating in opposite directions, summation was never observed. This supports the hypothesis that direction selective mechanisms are responsible for motion perception at threshold. Further analysis of the data produced estimates of the spatial frequency tuning of these mechanisms. A quantitative model is proposed to interpret the data, and it is suggested that flickering gratings are not decomposed into their moving components by the visual system.     

Optimizing visual motion perception during eye movements.

We usually perceive a stationary, stable world and we are able to correctly estimate the direction of heading from optic flow despite coherent visual motion induced by eye movements. This astonishing example of perceptual invariance results from a comparison of visual information with internal reference signals predicting the visual consequences of an eye movement. Here we demonstrate that the reference signal predicting the consequences of smooth-pursuit eye movements is continuously calibrated on the basis of direction-selective interactions between the pursuit motor command and the rotational flow induced by the eye movement, thereby minimizing imperfections of the reference signal and guaranteeing an ecologically optimal interpretation of visual motion.     

The hierarchy of directional interactions in visual motion processing

It is well known that context influences our perception of visual motion direction. For example, spatial and temporal context manipulations can be used to induce two well-known motion illusions: direction repulsion and the direction after-effect (DAE). Both result in inaccurate perception of direction when a moving pattern is either superimposed on (direction repulsion), or presented following adaptation to (DAE), another pattern moving in a different direction. Remarkable similarities in tuning characteristics suggest that common processes underlie the two illusions. What is not clear, however, is whether the processes driving the two illusions are expressions of the same or different neural substrates. Here we report two experiments demonstrating that direction repulsion and the DAE are, in fact, expressions of different neural substrates. Our strategy was to use each of the illusions to create a distorted perceptual representation upon which the mechanisms generating the other illusion could potentially operate. We found that the processes mediating direction repulsion did indeed access the distorted perceptual representation induced by the DAE. Conversely, the DAE was unaffected by direction repulsion. Thus parallels in perceptual phenomenology do not necessarily imply common neural substrates. Our results also demonstrate that the neural processes driving the DAE occur at an earlier stage of motion processing than those underlying direction repulsion.     

Cellular mechanisms for direction selectivity in the retina

Direction selectivity represents a fundamental computation found across multiple sensory systems. In the mammalian visual system, direction selectivity appears first in the retina, where excitatory and inhibitory interneurons release neurotransmitter most rapidly during movement in a preferred direction. Two parallel sets of interneuron signals are integrated by a direction-selective ganglion cell, which creates a direction preference for both bright and dark moving objects. Direction selectivity of synaptic input becomes amplified by action potentials in the ganglion cell dendrites. Recent work has elucidated direction-selective mechanisms in inhibitory circuitry, but mechanisms in excitatory circuitry remain unexplained.     

一种感知视觉运动信息的简化脑模型

视觉运动信息的感知过程,包括从局域运动检测到对模式整体运动的感知过程.我们以蝇视觉系统的图形-背景相对运动分辨的神经回路网络为基本框架,采用初级运动检测器的六角形阵列作为输入层,构造了一种感知视觉运动信息的简化脑模型,模拟了运动信息应该神经计算模型各个层次上的处理.该模型对差分行为实验结果作出了正确预测.本文并对空间生理整合的神经机制作了讨论.     

Mathematical analysis and modeling of motion direction selectivity in the retina

Motion detection is one of the most important and primitive computations performed by our visual system. Specifically in the retina, ganglion cells producing motion direction-selective responses have been addressed by different disciplines, such as mathematics, neurophysiology and computational modeling, since the beginnings of vision science. Although a number of studies have analyzed theoretical and mathematical considerations for such responses, a clear picture of the underlying cellular mechanisms is only recently emerging. In general, motion direction selectivity is based on a non-linear asymmetric computation inside a receptive field differentiating cell responses between preferred and null direction stimuli. To what extent can biological findings match these considerations? In this review, we outline theoretical and mathematical studies of motion direction selectivity, aiming to map the properties of the models onto the neural circuitry and synaptic connectivity found in the retina. Additionally, we review several compartmental models that have tried to fill this gap. Finally, we discuss the remaining challenges that computational models will have to tackle in order to fully understand the retinal motion direction-selective circuitry.     

Interocular motion combination for dichoptic moving stimuli

When gratings moving in different directions are presented separately to the two eyes, we typically perceive periods of the combination of motion in the two eyes as well as periods of one or the other monocular motions. To investigate whether such interocular motion combination is determined by the intersection-of-constraints (IOC) or vector average mechanism, we recorded both optokinetic nystagmus eye movements (OKN) and perception during dichoptic presentation of moving gratings and random-dot patterns with various differences of interocular motion direction. For moving gratings, OKN alternately tracks not only the direction of the two monocular motions but also the direction of their combined motion. The OKN in the combined motion direction is highly correlated with the perceived direction of combined motion; its velocity complies with the IOC rule rather than the vector average of the dichoptic motion stimuli. For moving random-dot patterns, both OKN and perceived motion alternate only between the directions of the two monocular motions. These results suggest that interocular motion combination in dichoptic gratings is determined by the IOC and depends on their form.     

Feature-based attention increases the selectivity of population responses in primate visual cortex

BACKGROUND: Attending to the spatial location or to nonspatial features of visual stimuli can modulate neuronal responses in primate visual cortex. The modulation by spatial attention changes the gain of sensory neurons and strengthens the representation of attended locations without changing neuronal selectivities such as directionality, i.e., the ratio of responses to preferred and anti-preferred directions of motion. Whether feature-based attention acts in a similar manner is unknown. RESULTS: To clarify this issue, we recorded the responses of 135 direction-selective neurons in the middle temporal area (MT) of two macaques to an unattended moving random dot pattern (the distractor) positioned inside a neuron's receptive field while the animals attended to a second moving pattern positioned in the opposite hemifield. Responses to different directions of the distractor were modulated by the same factor (approximately 12%) as long as the attended direction remained unchanged. On the other hand, systematically changing the attended direction from a neuron's preferred to its anti-preferred direction caused a systematic change of the attentional modulation from an enhancement to a suppression, increasing directionality by about 20%. CONCLUSIONS: The results show that (1) feature-based attention exerts a multiplicative modulation upon neuronal responses and that the strength of this modulation depends on the similarity between the attended feature and the cell's preferred feature, in line with the feature-similarity gain model, and (2) at the level of the neuronal population, feature-based attention increases the selectivity for attended features by increasing the responses of neurons preferring this feature value while decreasing responses of neurons tuned to the opposite feature value.     

A motion illusion reveals mechanisms of perceptual stabilization

Visual illusions are valuable tools for the scientific examination of the mechanisms underlying perception. In the peripheral drift illusion special drift patterns appear to move although they are static. During fixation small involuntary eye movements generate retinal image slips which need to be suppressed for stable perception. Here we show that the peripheral drift illusion reveals the mechanisms of perceptual stabilization associated with these micromovements. In a series of experiments we found that illusory motion was only observed in the peripheral visual field. The strength of illusory motion varied with the degree of micromovements. However, drift patterns presented in the central (but not the peripheral) visual field modulated the strength of illusory peripheral motion. Moreover, although central drift patterns were not perceived as moving, they elicited illusory motion of neutral peripheral patterns. Central drift patterns modulated illusory peripheral motion even when micromovements remained constant. Interestingly, perceptual stabilization was only affected by static drift patterns, but not by real motion signals. Our findings suggest that perceptual instabilities caused by fixational eye movements are corrected by a mechanism that relies on visual rather than extraretinal (proprioceptive or motor) signals, and that drift patterns systematically bias this compensatory mechanism. These mechanisms may be revealed by utilizing static visual patterns that give rise to the peripheral drift illusion, but remain undetected with other patterns. Accordingly, the peripheral drift illusion is of unique value for examining processes of perceptual stabilization.     

Two-dimensional substructure of MT receptive fields.

Neurons at progressively higher levels of the visual system have progressively larger, more complicated receptive fields, presumably constructed from simpler antecedent receptive fields. To study this hierarchical organization, we used sparse white noise to map receptive-field substructure (second order Wiener-like kernels) in an extrastriate motion processing area (MT) of alert monkeys. The maps revealed a clear substructure, on a spatial scale comparable to the receptive fields of the V1 inputs. There were both facilitatory and suppressive interactions that differed in spatial organization and time course. Directional interactions were remarkably precise over a very small spatial range, and reversed when successive stimuli reversed contrast--a neural correlate of "reverse phi" motion perception. The maps of some cells had an unexpected, curved shape, which challenges existing models for direction selectivity.     

Direct evidence for encoding of motion streaks in human visual cortex

Temporal integration in the visual system causes fast-moving objects to generate static, oriented traces (‘motion streaks’), which could be used to help judge direction of motion. While human psychophysics and single-unit studies in non-human primates are consistent with this hypothesis, direct neural evidence from the human cortex is still lacking. First, we provide psychophysical evidence that faster and slower motions are processed by distinct neural mechanisms: faster motion raised human perceptual thresholds for static orientations parallel to the direction of motion, whereas slower motion raised thresholds for orthogonal orientations. We then used functional magnetic resonance imaging to measure brain activity while human observers viewed either fast (‘streaky’) or slow random dot stimuli moving in different directions, or corresponding static-oriented stimuli. We found that local spatial patterns of brain activity in early retinotopic visual cortex reliably distinguished between static orientations. Critically, a multivariate pattern classifier trained on brain activity evoked by these static stimuli could then successfully distinguish the direction of fast (‘streaky’) but not slow motion. Thus, signals encoding static-oriented streak information are present in human early visual cortex when viewing fast motion. These experiments show that motion streaks are present in the human visual system for faster motion.     

Neural population representation hypothesis of visual flow and its illusory after effect in the brain: psychophysics,neurophysiology and computational approaches

The neural representation of motion aftereffects induced by various visual flows (translational, rotational, motion-in-depth, and translational transparent flows) was studied under the hypothesis that the imbalances in discharge activities would occur in favor in the direction opposite to the adapting stimulation in the monkey MST cells (cells in the medial superior temporal area) which can discriminate the mode (i.e., translational, rotational, or motion-in-depth) of the given flow. In single-unit recording experiments conducted on anaesthetized monkeys, we found that the rate of spontaneous discharge and the sensitivity to a test stimulus moving in the preferred direction decreased after receiving an adapting stimulation moving in the preferred direction, whereas they increased after receiving an adapting stimulation moving in the null direction. To consistently explain the bidirectional perception of a transparent visual flow and its unidirectional motion aftereffect by the same hypothesis, we need to assume the existence of two subtypes of MST D cells which show directionally selective responses to a translational flow: component cells and integration cells. Our physiological investigation revealed that the MST D cells could be divided into two types: one responded to a transparent flow by two peaks at the instances when the direction of one of the component flow matched the preferred direction of the cell, and the other responded by a single peak at the instance when the direction of the integrated motion matched the preferred direction. In psychophysical experiments on human subjects, we found evidence for the existence of component and integration representations in the human brain. To explain the different motion perceptions, i.e., two transparent flows during presentation of the flows and a single flow in the opposite direction to the integrated flows after stopping the flow stimuli, we suggest that the pattern-discrimination system can select the motion representation that is consistent with the perception of the pattern from two motion representations. We discuss the computational aspects related to the integration of component motion fields.     

Motion detection in human vision: a unifying approach based on energy and features

Most studies of human motion perception have been based on the implicit assumption that the brain has only one motion-detection system, or at least that only one is operational in any given instance. We show, in the context of direction perception in spatially filtered two-frame random-dot kinematograms, that two quite different mechanisms operate simultaneously in the detection of such patterns. One mechanism causes reversal of the perceived direction (reversed-phi motion) when the image contrast is reversed between frames, and is highly dependent on the spatial-frequency content of the image. These characteristics are both signatures of detection based on motion energy. The other mechanism does not produce reversed-phi motion and is unaffected by spatial filtering. This appears to involve the tracking of unsigned complex spatial features. The perceived direction of a filtered dot pattern typically reflects a mixture of the two types of behaviour in any given instance. Although both types of mechanism have previously been invoked to explain the perception of motion of different types of image, the simultaneous involvement of two mechanisms in the detection of the same simple rigid motion of a pattern suggests that motion perception in general results from a combination of mechanisms working simultaneously on different principles in the same circumstances.     

The visual processing of motion-defined transparency

Our understanding of how the visual system processes motion transparency, the phenomenon by which multiple directions of motion are perceived to coexist in the same spatial region, has grown considerably in the past decade. There is compelling evidence that the process is driven by global-motion mechanisms. Consequently, although transparently moving surfaces are readily segmented over an extended space, the visual system cannot separate two motion signals that coexist in the same local region. A related issue is whether the visual system can detect transparently moving surfaces simultaneously or whether the component signals encounter a serial 'bottleneck' during their processing. Our initial results show that, at sufficiently short stimulus durations, observers cannot accurately detect two superimposed directions; yet they have no difficulty in detecting one pattern direction in noise, supporting the serial-bottleneck scenario. However, in a second experiment, the difference in performance between the two tasks disappears when the component patterns are segregated. This discrepancy between the processing of transparent and non-overlapping patterns may be a consequence of suppressed activity of global-motion mechanisms when the transparent surfaces are presented in the same depth plane. To test this explanation, we repeated our initial experiment while separating the motion components in depth. The marked improvement in performance leads us to conclude that transparent motion signals are represented simultaneously.     

Influence of Visual Motion,Suggestion, and Illusory Motion on Self-Motion Perception in the Horizontal Plane

A moving visual field can induce the feeling of self-motion or vection. Illusory motion from static repeated asymmetric patterns creates a compelling visual motion stimulus, but it is unclear if such illusory motion can induce a feeling of self-motion or alter self-motion perception. In these experiments, human subjects reported the perceived direction of self-motion for sway translation and yaw rotation at the end of a period of viewing set visual stimuli coordinated with varying inertial stimuli. This tested the hypothesis that illusory visual motion would influence self-motion perception in the horizontal plane. Trials were arranged into 5 blocks based on stimulus type: moving star field with yaw rotation, moving star field with sway translation, illusory motion with yaw, illusory motion with sway, and static arrows with sway. Static arrows were used to evaluate the effect of cognitive suggestion on self-motion perception. Each trial had a control condition; the illusory motion controls were altered versions of the experimental image, which removed the illusory motion effect. For the moving visual stimulus, controls were carried out in a dark room. With the arrow visual stimulus, controls were a gray screen. In blocks containing a visual stimulus there was an 8s viewing interval with the inertial stimulus occurring over the final 1s. This allowed measurement of the visual illusion perception using objective methods. When no visual stimulus was present, only the 1s motion stimulus was presented. Eight women and five men (mean age 37) participated. To assess for a shift in self-motion perception, the effect of each visual stimulus on the self-motion stimulus (cm/s) at which subjects were equally likely to report motion in either direction was measured. Significant effects were seen for moving star fields for both translation (p = 0.001) and rotation (p<0.001), and arrows (p = 0.02). For the visual motion stimuli, inertial motion perception was shifted in the direction consistent with the visual stimulus. Arrows had a small effect on self-motion perception driven by a minority of subjects. There was no significant effect of illusory motion on self-motion perception for either translation or rotation (p>0.1 for both). Thus, although a true moving visual field can induce self-motion, results of this study show that illusory motion does not.     

猫前内侧上雪氏区视神经元对运动随机线条的反应

对于运动信息在脑内的加工,一种观点认为分两阶段进行,低级视皮层只对运动图形内部成分的取向进行调谐,高级视皮层整合低级视皮层的输入,对图形整体的运动方向敏感。用网格（plaid）作为刺激的实验表明,在较低级皮层区,细胞多表现为成分方向选择性（Component-motion Selectivity),即对刺激中的取向因素敏感：而较高视皮层的细胞多表现为整体方向选择性（Pattern-motion Selecitivity),对运动整体的方向敏感,从而支持运动信息加工的“两阶段”理论。实验中,用一系列运动随机线条刺激（random line patterns)。研究猫前内侧上雪氏区（Anteriormedial lateral suprasylvian area,AMLS)神经元的方向调谐特性。结果表明多数细胞为整体方向选择性,且随线长增加此类细胞比例下降,而成分方向选择性细胞的比例有所增加,呈现由整体方向选择性向中间类型（Unclassified),由中间类型向成分方向选择性变化的趋势,提示整体或成分方向选择性可能并非细胞的固有特性,而是可以随刺激取向因素的变化而改变的。     

Noninformative vision improves haptic spatial perception

Previous studies have attempted to map somatosensory space via haptic matching tasks and have shown that individuals make large and systematic matching errors, the magnitude and angular direction of which vary systematically through the workspace. Based upon such demonstrations, it has been suggested that haptic space is non-Euclidian. This conclusion assumes that spatial perception is modality specific, and it largely ignores the fact that tactile matching tasks involve active, exploratory arm movements. Here we demonstrate that, when individuals match two bar stimuli (i.e., make them parallel) in circumstances favoring extrinsic (visual) coordinates, providing noninformative visual information significantly increases the accuracy of haptic perception. In contrast, when individuals match the same bar stimuli in circumstances favoring the coding of movements in intrinsic (limb-based) coordinates, providing identical noninformative visual information either has no effect or leads to the decreased accuracy of haptic perception. These results are consistent with optimal integration models of sensory integration in which the weighting given to visual and somatosensory signals depends upon the precision of the visual and somatosensory information and provide important evidence for the task-dependent integration of visual and somatosensory signals during the construction of a representation of peripersonal space.