Reconstructing the world: switching from segmentation to integration allows neurons in area MT to make "sense" of the visual scene

Huang et al. in this issue of Neuron show that primate area MT neurons exploit contextual cues to adequately interpret motion information. MT neurons switch from segmentation to integration when motion arises from single rather than multiple objects. This switching may help solve the aperture problem and bind distant object components into a perceptual whole.     

Defining the V5/MT neuronal pool for perceptual decisions in a visual stereo-motion task

In the primate visual cortex, neurons signal differences in the appearance of objects with high precision. However, not all activated neurons contribute directly to perception. We defined the perceptual pool in extrastriate visual area V5/MT for a stereo-motion task, based on trial-by-trial co-variation between perceptual decisions and neuronal firing (choice probability (CP)). Macaque monkeys were trained to discriminate the direction of rotation of a cylinder, using the binocular depth between the moving dots that form its front and rear surfaces. We manipulated the activity of single neurons trial-to-trial by introducing task-irrelevant stimulus changes: dot motion in cylinders was aligned with neuronal preference on only half the trials, so that neurons were strongly activated with high firing rates on some trials and considerably less activated on others. We show that single neurons maintain high neurometric sensitivity for binocular depth in the face of substantial changes in firing rate. CP was correlated with neurometric sensitivity, not level of activation. In contrast, for individual neurons, the correlation between perceptual choice and neuronal activity may be fundamentally different when responding to different stimulus versions. Therefore, neuronal pools supporting sensory discrimination must be structured flexibly and independently for each stimulus configuration to be discriminated.This article is part of the themed issue ‘Vision in our three-dimensional world''.     

Directional anisotropies reveal a functional segregation of visual motion processing for perception and action

Human exhibits an anisotropy in direction perception: discrimination is superior when motion is around horizontal or vertical rather than diagonal axes. In contrast to the consistent directional anisotropy in perception, we found only small idiosyncratic anisotropies in smooth pursuit eye movements, a motor action requiring accurate discrimination of visual motion direction. Both pursuit and perceptual direction discrimination rely on signals from the middle temporal visual area (MT), yet analysis of multiple measures of MT neuronal responses in the macaque failed to provide evidence of a directional anisotropy. We conclude that MT represents different motion directions uniformly, and subsequent processing creates a directional anisotropy in pathways unique to perception. Our data support the hypothesis that, at least for visual motion, perception and action are guided by inputs from separate sensory streams. The directional anisotropy of perception appears to originate after the two streams have segregated and downstream from area MT.     

A common neuronal code for perceptual processes in visual cortex? Comparing choice and attentional correlates in V5/MT.

In the past two decades, sensory neuroscience has moved from describing response properties to external stimuli in cerebral cortex to establishing connections between neuronal activity and sensory perception. The seminal studies by Newsome, Movshon and colleagues in the awake behaving macaque firmly link single cells in extrastriate area V5/MT and perception of motion. A decade later, extrastriate visual cortex appears awash with neuronal correlates for many different perceptual tasks. Examples are attentional signals, choice signals for ambiguous images, correlates for binocular rivalry, stereo and shape perception, and so on. These diverse paradigms are aimed at elucidating the neuronal code for perceptual processes, but it has been little studied how they directly compare or even interact. In this paper, I explore to what degree the measured neuronal signals in V5/MT for choice and attentional paradigms might reflect a common neuronal mechanism for visual perception.     

Neuronal activity and its links with the perception of multi-stable figures

In order to isolate the neuronal activity that relates to the making of perceptual decisions, we have made use of a perceptually ambiguous motion stimulus. This stimulus lies on the boundary between two perceptual categories that correspond to clockwise and counter-clockwise rotation of a three-dimensional figure. It consists of a two-dimensional pattern of moving dots that are capable of generating these two, distinct, three-dimensional percepts. We have studied the responses of neurons in cortical area V5/MT whilst macaque monkeys report judgements about the perceptual configuration of this stimulus. We extract a quantitative statistic called 'choice probability' that expresses the covariation of neuronal activity and perceptual choice. An analysis of choice probabilities shows that the pool of neurons involved in the perceptual decisions is a tightly constrained subset of the population of sensory neurons relevant to the perceptual task.     

Attention to surfaces modulates motion processing in extrastriate area MT

In the visual system, early atomized representations are grouped into higher-level entities through processes of perceptual organization. Here we present neurophysiological evidence that a representation of a simple object, a surface defined by color and motion, can be the unit of attentional selection at an early stage of visual processing. Monkeys were cued by the color of a fixation spot to attend to one of two transparent random-dot surfaces, one red and one green, which occupied the same region of space. Motion of the attended surface drove neurons in the middle temporal (MT) visual area more strongly than physically identical motion of the non-attended surface, even though both occurred within the spotlight of attention. Surface-based effects of attention persisted even without differential surface coloring, but attentional modulation was stronger with color. These results show that attention can select surface representations to modulate visual processing as early as cortical area MT.     

Two-dimensional substructure of stereo and motion interactions in macaque visual cortex

The analysis of object motion and stereoscopic depth are important tasks that are begun at early stages of the primate visual system. Using sparse white noise, we mapped the receptive field substructure of motion and disparity interactions in neurons in V1 and MT of alert monkeys. Interactions in both regions revealed subunits similar in structure to V1 simple cells. For both motion and stereo, the scale and shape of the receptive field substructure could be predicted from conventional tuning for bars or dot-field stimuli, indicating that the small-scale interactions were repeated across the receptive fields. We also found neurons in V1 and in MT that were tuned to combinations of spatial and temporal binocular disparities, suggesting a possible neural substrate for the perceptual Pulfrich phenomenon. Our observations constrain computational and developmental models of motion-stereo integration.     

Neural correlates of contrast detection at threshold

Human psychophysical studies have demonstrated that, for stimuli near the threshold of visibility, detection of motion in one direction is unaffected by the superimposition of motion in the opposite direction. To investigate the neural basis for this perceptual phenomenon, we recorded from directionally selective neurons in macaque visual area MT (middle temporal visual area). Contrast thresholds obtained for single gratings moving in a neuron's preferred direction were compared with those obtained for motion presented simultaneously in the neuron's preferred and antipreferred directions. A simple model based on probability summation between neurons tuned to opposite directions could sufficiently account for contrast thresholds revealed psychophysically, suggesting that area MT is likely to provide the neural basis for contrast detection of stimuli modulated in time.     

Multivoxel Pattern Analysis Reveals Auditory Motion Information in MT+ of Both Congenitally Blind and Sighted Individuals

Cross-modal plasticity refers to the recruitment of cortical regions involved in the processing of one modality (e.g. vision) for processing other modalities (e.g. audition). The principles determining how and where cross-modal plasticity occurs remain poorly understood. Here, we investigate these principles by testing responses to auditory motion in visual motion area MT+ of congenitally blind and sighted individuals. Replicating previous reports, we find that MT+ as a whole shows a strong and selective responses to auditory motion in congenitally blind but not sighted individuals, suggesting that the emergence of this univariate response depends on experience. Importantly, however, multivoxel pattern analyses showed that MT+ contained information about different auditory motion conditions in both blind and sighted individuals. These results were specific to MT+ and not found in early visual cortex. Basic sensitivity to auditory motion in MT+ is thus experience-independent, which may be a basis for the region''s strong cross-modal recruitment in congenital blindness.     

Complementary and dynamic type II cadherin expression associated with development of the primate visual system

The middle temporal visual area (MT, also known as V5) is a visual association area that is particularly evolved in the primate brain. The MT receives input from the primary visual area (V1), constitutes part of the dorsal visual pathway, and plays an essential role in processing motion. Connections between the MT and V1 in the primate brain are formed after birth, and are related to the maturation of visual system. However, it remains to be determined what molecular mechanisms control the formation and maturation of the visual system. Cadherins are transmembrane proteins, originally isolated as cell adhesion molecules, which have multiple roles in synapse formation and function. To investigate potential involvement of cadherins in development of the primate visual system, we examined type II cadherin expression (cadherin‐6, ‐8, ‐12) in cortical and thalamic visual areas of pre‐ and postnatal brains of the common marmoset (Callithrix jacchus). In the prenatal brain, cadherin‐6 was dominantly expressed in the pulvino‐MT pathway whereas cadherin‐8 was dominant in the lateral geniculate nucleus (LGN)‐V1 pathway. During postnatal development, there was a downregulation of cadherin‐6 and upregulation of cadherin‐8 expression in the MT. The timing of this cadherin exchange preceded the development of V1‐MT connections. Our results suggest the possibility that changes in cadherin expression are involved in the development of the primate visual system, and that a switch in cadherin expression may be a general mechanism to control neural plasticity of highly cognitive abilities.     

Perceptual Learning of Motion Direction Discrimination with Suppressed and Unsuppressed MT in Humans: An fMRI Study

The middle temporal area of the extrastriate visual cortex (area MT) is integral to motion perception and is thought to play a key role in the perceptual learning of motion tasks. We have previously found, however, that perceptual learning of a motion discrimination task is possible even when the training stimulus contains locally balanced, motion opponent signals that putatively suppress the response of MT. Assuming at least partial suppression of MT, possible explanations for this learning are that 1) training made MT more responsive by reducing motion opponency, 2) MT remained suppressed and alternative visual areas such as V1 enabled learning and/or 3) suppression of MT increased with training, possibly to reduce noise. Here we used fMRI to test these possibilities. We first confirmed that the motion opponent stimulus did indeed suppress the BOLD response within hMT+ compared to an almost identical stimulus without locally balanced motion signals. We then trained participants on motion opponent or non-opponent stimuli. Training with the motion opponent stimulus reduced the BOLD response within hMT+ and greater reductions in BOLD response were correlated with greater amounts of learning. The opposite relationship between BOLD and behaviour was found at V1 for the group trained on the motion-opponent stimulus and at both V1 and hMT+ for the group trained on the non-opponent motion stimulus. As the average response of many cells within MT to motion opponent stimuli is the same as their response to non-directional flickering noise, the reduced activation of hMT+ after training may reflect noise reduction.     

Human perceptual and metacognitive decision-making rely on distinct brain networks

Perceptual decisions depend on the ability to exploit available sensory information in order to select the most adaptive option from a set of alternatives. Such decisions depend on the perceptual sensitivity of the organism, which is generally accompanied by a corresponding level of certainty about the choice made. Here, by use of corticocortical paired associative transcranial magnetic stimulation protocol (ccPAS) aimed at inducing plastic changes, we shaped perceptual sensitivity and metacognitive ability in a motion discrimination task depending on the targeted network, demonstrating their functional dissociation. Neurostimulation aimed at boosting V5/MT+-to-V1/V2 back-projections enhanced motion sensitivity without impacting metacognition, whereas boosting IPS/LIP-to-V1/V2 back-projections increased metacognitive efficiency without impacting motion sensitivity. This double-dissociation provides causal evidence of distinct networks for perceptual sensitivity and metacognitive ability in humans.

Transcranial magnetic stimulation targeting cortico-cortical connections reveals a functional dissociation between temporo-visual and parieto-visual re-entrant pathways in humans, controlling perceptual sensitivity and metacognitive abilities respectively, during a visual motion perception task.     

Color signals in area MT of the macaque monkey

The relationship between the neural processing of color and motion information has been a contentious issue in visual neuroscience. We examined this relationship directly by measuring neural responses to isoluminant S cone signals in extrastriate area MT of the macaque monkey. S cone stimuli produced robust, direction-selective responses at most recording sites, indicating that color signals are present in MT. While these responses were unequivocal, S cone contrast sensitivity was, on average, 1.0-1.3 log units lower than luminance contrast sensitivity. The presence of S cone responses and the relative sensitivity of MT neurons to S cone and luminance signals agree with functional magnetic resonance imaging (fMRI) measurements in human MT+. The results are consistent with the hypothesis that color signals in MT influence behavior in speed judgment tasks.     

采用Reichardt运动检测器和Boltzmann Machine神经网络的运动计算

建立了一个探讨灵长类视皮层从Ｖ１区到ＭＴ区的运动信息加工原理的计算模型,这个过程的突出特征是视觉运动信息经过了从局部检测进步到整体感知。模型的第一层由用于抽提运动模式的局部速度以及结构性质的Ｒｅｉｃｈａｒｄｔ运动检测器组成,进一步的加工是通过Ｂｏｌｔｚｍａｎｎ Ｍａｃｈｉｎｅ神经网络来实现的。这种网络的学习算法具有局部更新的显著性质,在学习阶段,网络不断地修改联结权重以形成对于记录在网络的显单元上     

A Bio-Inspired,Motion-Based Analysis of Crowd Behavior Attributes Relevance to Motion Transparency,Velocity Gradients,and Motion Patterns

The analysis of motion crowds is concerned with the detection of potential hazards for individuals of the crowd. Existing methods analyze the statistics of pixel motion to classify non-dangerous or dangerous behavior, to detect outlier motions, or to estimate the mean throughput of people for an image region. We suggest a biologically inspired model for the analysis of motion crowds that extracts motion features indicative for potential dangers in crowd behavior. Our model consists of stages for motion detection, integration, and pattern detection that model functions of the primate primary visual cortex area (V1), the middle temporal area (MT), and the medial superior temporal area (MST), respectively. This model allows for the processing of motion transparency, the appearance of multiple motions in the same visual region, in addition to processing opaque motion. We suggest that motion transparency helps to identify “danger zones” in motion crowds. For instance, motion transparency occurs in small exit passages during evacuation. However, motion transparency occurs also for non-dangerous crowd behavior when people move in opposite directions organized into separate lanes. Our analysis suggests: The combination of motion transparency and a slow motion speed can be used for labeling of candidate regions that contain dangerous behavior. In addition, locally detected decelerations or negative speed gradients of motions are a precursor of danger in crowd behavior as are globally detected motion patterns that show a contraction toward a single point. In sum, motion transparency, image speeds, motion patterns, and speed gradients extracted from visual motion in videos are important features to describe the behavioral state of a motion crowd.     

Delayed perceptual awareness in rapid perceptual decisions

The flourishing of studies on the neural correlates of decision-making calls for an appraisal of the relation between perceptual decisions and conscious perception. By exploiting the long integration time of noisy motion stimuli, and by forcing human observers to make difficult speeded decisions--sometimes a blind guess--about stimulus direction, we traced the temporal buildup of motion discrimination capability and perceptual awareness, as assessed trial by trial through direct rating. We found that both increased gradually with motion coherence and viewing time, but discrimination was systematically leading awareness, reaching a plateau much earlier. Sensitivity and criterion changes contributed jointly to the slow buildup of perceptual awareness. It made no difference whether motion discrimination was accomplished by saccades or verbal responses. These findings suggest that perceptual awareness emerges on the top of a developing or even mature perceptual decision. We argue that the middle temporal (MT) cortical region does not confer us the full phenomenic depth of motion perception, although it may represent a precursor stage in building our subjective sense of visual motion.     

Breaking cover: neural responses to slow and fast camouflage-breaking motion

Primates need to detect and recognize camouflaged animals in natural environments. Camouflage-breaking movements are often the only visual cue available to accomplish this. Specifically, sudden movements are often detected before full recognition of the camouflaged animal is made, suggesting that initial processing of motion precedes the recognition of motion-defined contours or shapes. What are the neuronal mechanisms underlying this initial processing of camouflaged motion in the primate visual brain? We investigated this question using intrinsic-signal optical imaging of macaque V1, V2 and V4, along with computer simulations of the neural population responses. We found that camouflaged motion at low speed was processed as a direction signal by both direction- and orientation-selective neurons, whereas at high-speed camouflaged motion was encoded as a motion-streak signal primarily by orientation-selective neurons. No population responses were found to be invariant to the camouflage contours. These results suggest that the initial processing of camouflaged motion at low and high speeds is encoded as direction and motion-streak signals in primate early visual cortices. These processes are consistent with a spatio-temporal filter mechanism that provides for fast processing of motion signals, prior to full recognition of camouflage-breaking animals.     

A Network Model of Motion Processing in Area MT of Primates

A simple and biologically plausible model is proposed to simulatethe visual motion processing taking place in the middle temporal (MT) areaof the visual cortex in the primate brain. The model is ahierarchical neural network composed of multiple competitive learninglayers. The input layer of the network simulates the neurons in the primaryvisual cortex (V1), which are sensitive to the orientation and motionvelocity of the visual stimuli, and the middle and output layers of thenetwork simulate the component MT and pattern MT neurons, which areselectively responsive to local and global motions, respectively. Thenetwork model was tested with various simulated motion patterns (random dotsof different direction correlations, transparent motion, grating and plaidpatterns, and so on). The response properties of the model closely resemblemany of the known features of the MT neurons found neurophysiologically.These results show that the sophisticated response behaviors of the MTneurons can emerge naturally from some very simple models, such as acompetitive learning network.     

Visual perception: mind and brain see eye to eye

Recent functional imaging studies have identified neural activity that is closely associated with the perception of illusory motion. The mapping of the mind onto the bin appears to be one-to-one: activity in visual 'motion area' MT is highly correlated with perceptual experience.     

Laminar cortical dynamics of visual form and motion interactions during coherent object motion perception

How do visual form and motion processes cooperate to compute object motion when each process separately is insufficient? Consider, for example, a deer moving behind a bush. Here the partially occluded fragments of motion signals available to an observer must be coherently grouped into the motion of a single object. A 3D FORMOTION model comprises five important functional interactions involving the brain's form and motion systems that address such situations. Because the model's stages are analogous to areas of the primate visual system, we refer to the stages by corresponding anatomical names. In one of these functional interactions, 3D boundary representations, in which figures are separated from their backgrounds, are formed in cortical area V2. These depth-selective V2 boundaries select motion signals at the appropriate depths in MT via V2-to-MT signals. In another, motion signals in MT disambiguate locally incomplete or ambiguous boundary signals in V2 via MT-to-V1-to-V2 feedback. The third functional property concerns resolution of the aperture problem along straight moving contours by propagating the influence of unambiguous motion signals generated at contour terminators or corners. Here, sparse 'feature tracking signals' from, for example, line ends are amplified to overwhelm numerically superior ambiguous motion signals along line segment interiors. In the fourth, a spatially anisotropic motion grouping process takes place across perceptual space via MT-MST feedback to integrate veridical feature-tracking and ambiguous motion signals to determine a global object motion percept. The fifth property uses the MT-MST feedback loop to convey an attentional priming signal from higher brain areas back to V1 and V2. The model's use of mechanisms such as divisive normalization, endstopping, cross-orientation inhibition, and long-range cooperation is described. Simulated data include: the degree of motion coherence of rotating shapes observed through apertures, the coherent vs. element motion percepts separated in depth during the chopsticks illusion, and the rigid vs. nonrigid appearance of rotating ellipses.