Polymodal motion processing in posterior parietal and premotor cortex: a human fMRI study strongly implies equivalencies between humans and monkeys

In monkeys, posterior parietal and premotor cortex play an important integrative role in polymodal motion processing. In contrast, our understanding of the convergence of senses in humans is only at its beginning. To test for equivalencies between macaque and human polymodal motion processing, we used functional MRI in normals while presenting moving visual, tactile, or auditory stimuli. Increased neural activity evoked by all three stimulus modalities was found in the depth of the intraparietal sulcus (IPS), ventral premotor, and lateral inferior postcentral cortex. The observed activations strongly suggest that polymodal motion processing in humans and monkeys is supported by equivalent areas. The activations in the depth of IPS imply that this area constitutes the human equivalent of macaque area VIP.     

See me, hear me, touch me: multisensory integration in lateral occipital-temporal cortex

Our understanding of multisensory integration has advanced because of recent functional neuroimaging studies of three areas in human lateral occipito-temporal cortex: superior temporal sulcus, area LO and area MT (V5). Superior temporal sulcus is activated strongly in response to meaningful auditory and visual stimuli, but responses to tactile stimuli have not been well studied. Area LO shows strong activation in response to both visual and tactile shape information, but not to auditory representations of objects. Area MT, an important region for processing visual motion, also shows weak activation in response to tactile motion, and a signal that drops below resting baseline in response to auditory motion. Within superior temporal sulcus, a patchy organization of regions is activated in response to auditory, visual and multisensory stimuli. This organization appears similar to that observed in polysensory areas in macaque superior temporal sulcus, suggesting that it is an anatomical substrate for multisensory integration. A patchy organization might also be a neural mechanism for integrating disparate representations within individual sensory modalities, such as representations of visual form and visual motion.     

Integration of Motion Responses Underlying Directional Motion Anisotropy in Human Early Visual Cortical Areas

Recent imaging studies have reported directional motion biases in human visual cortex when perceiving moving random dot patterns. It has been hypothesized that these biases occur as a result of the integration of motion detector activation along the path of motion in visual cortex. In this study we investigate the nature of such motion integration with functional MRI (fMRI) using different motion stimuli. Three types of moving random dot stimuli were presented, showing either coherent motion, motion with spatial decorrelations or motion with temporal decorrelations. The results from the coherent motion stimulus reproduced the centripetal and centrifugal directional motion biases in V1, V2 and V3 as previously reported. The temporally decorrelated motion stimulus resulted in both centripetal and centrifugal biases similar to coherent motion. In contrast, the spatially decorrelated motion stimulus resulted in small directional motion biases that were only present in parts of visual cortex coding for higher eccentricities of the visual field. In combination with previous results, these findings indicate that biased motion responses in early visual cortical areas most likely depend on the spatial integration of a simultaneously activated motion detector chain.     

Representation of motion onset and offset in an augmented Barlow-Levick model of motion detection

Kinetic occlusion produces discontinuities in the optic flow field, whose perception requires the detection of an unexpected onset or offset of otherwise predictably moving or stationary contrast patches. Many cells in primate visual cortex are directionally selective for moving contrasts, and recent reports suggest that this selectivity arises through the inhibition of contrast signals moving in the cells’ null direction, as in the rabbit retina. This nulling inhibition circuit (Barlow-Levick) is here extended to also detect motion onsets and offsets. The selectivity of extended circuit units, measured as a peak evidence accumulation response to motion onset/offset compared to the peak response to constant motion, is analyzed as a function of stimulus speed. Model onset cells are quiet during constant motion, but model offset cells activate during constant motion at slow speeds. Consequently, model offset cell speed tuning is biased towards higher speeds than onset cell tuning, similarly to the speed tuning of cells in the middle temporal area when exposed to speed ramps. Given a population of neurons with different preferred speeds, this asymmetry addresses a behavioral paradox—why human subjects in a simple reaction time task respond more slowly to motion offsets than onsets for low speeds, even though monkey neuron firing rates react more quickly to the offset of a preferred stimulus than to its onset.     

A linear model for symmetric receptive fields: implications for classification tests with flashed and moving images

The purpose of this study was to explore the effects of spatial and temporal properties on the expected responses of visual neurons that have linear receptive fields (RFs), particularly those having a mirror symmetric distribution of spatial subregions. Receptive fields that are symmetric in at least one spatial dimension occur in neurons of the retina, the lateral geniculate nucleus (LGN), and the visual cortex of mammals. Responses to flashing bars, moving bars, and moving edges were studied for different configurations of an analog RF model in which spatial and temporal aspects were varied independently. Responses of the model at intermediate stimulus speeds were found to agree with responses in the literature for X and Y units of the LGN and often for simple units of the visual cortex. In particular, having separated regions of response to light and dark edges, an identifying property of simple cells, was found to be a linear consequence of RF regions responding inversely to stimuli of opposite polarity. Model differences from responses of cortical complex units show that a linear model cannot mimic their responses, and imply that complex units employ major nonlinearities in coding image polarity (light vs dark), which signifies a nonlinearity in coding intensity. Because sudden flux changes inherent in flashing bars test mainly temporal RF properties, and slowly moving edges test mainly spatial properties, these two tests form a useful minimal set with which to describe and classify RFs. The usefulness of this set derives both from its sensitivity to spatial and temporal variables, and from the correlation between the linearity of a cell's processing of stimulus intensity and its RF classification.     

Cue-triggered activity replay in human early visual cortex

The recall of learned temporal sequences by a visual cue is an important form of experience-based neural plasticity. Here we observed such reactivation in awake human visual cortex using intracranial recording. After repeated exposure to a moving dot, a flash of the dot was able to trigger neural reactivation in the downstream receptive field along the motion path. This effect was observed only when the cue appeared near the receptive field. The estimated traveling speed was faster compared to the activation induced by the real motion. We suggest a range-limited, time-compressed reactivation as a result of repeated visual exposure in awake human visual cortex.     

Distributed neural systems for the generation of visual images

Visual perception of houses, faces, and chairs evoke differential responses in ventral temporal cortex. Using fMRI, we compared activations evoked by perception and imagery of these object categories. We found content-related activation during imagery in extrastriate cortex, but this activity was restricted to small subsets of the regions that showed category-related activation during perception. Within ventral temporal cortex, activation during imagery evoked stronger responses on the left whereas perception evoked stronger responses on the right. Additionally, visual imagery evoked activity in parietal and frontal cortex, but this activity was not content related. These results suggest that content-related activation during imagery in visual extrastriate cortex may be implemented by "top-down" mechanisms in parietal and frontal cortex that mediate the retrieval of face and object representations from long-term memory and their maintenance through visual imagery.     

Brain potentials associated with pattern displacement and saccadic eye movements in humans and rhesus monkeys

Visual evoked potentials (VEPs) to the onset of motion of visual patterns and brain responses associated with saccadic eye movements (SRPs) were compared in human subjects and in rhesus monkeys. Three different velocities of pattern motion were employed. In humans, brain responses were recorded from six scalp areas. In monkeys, transcortical recordings were obtained from chronically implanted electrodes in the occipital, temporo-parietal, and frontal areas. In humans there was a clear difference in VEPs to the pattern motion between the anterior (Fz, Cz) and posterior (Pz, Oz) scalp regions. The earliest component was a positive peak at 85 ms at Oz followed by a negativity around 110 ms. In the fronto-central leads the VEP was characterized by a negativity at 145 ms and a subsequent broad positive component around 250 ms. SRP responses differed in the early components from the VEPs to pattern motion but a good correspondence was found in the morphology of the late components of the two types of brain potentials. Furthermore, flashed-on VEPs and SRPs elicited a late positivity of more pronounced amplitude than VEPs to pattern displacement. In monkeys similar findings were found: an early negative component of the pattern-displacement VEP could not be observed in the SRP responses over the visual cortex while the late portion of the SRP waveform was greater than the late positivity of the VEP to motion-onset.     

Competing neural responses for auditory and visual decisions

Why is it hard to divide attention between dissimilar activities, such as reading and listening to a conversation? We used functional magnetic resonance imaging (fMRI) to study interference between simple auditory and visual decisions, independently of motor competition. Overlapping activity for auditory and visual tasks performed in isolation was found in lateral prefrontal regions, middle temporal cortex and parietal cortex. When the visual stimulus occurred during the processing of the tone, its activation in prefrontal and middle temporal cortex was suppressed. Additionally, reduced activity was seen in modality-specific visual cortex. These results paralleled impaired awareness of the visual event. Even without competing motor responses, a simple auditory decision interferes with visual processing on different neural levels, including prefrontal cortex, middle temporal cortex and visual regions.     

综述: 初级视皮层的亮度信息加工

亮度(luminance)是最基本的视觉信息.与其他视觉特征相比,由于视神经元对亮度刺激的反应较弱,并且许多神经元对均匀亮度无反应,对亮度信息编码的神经机制知之甚少.初级视皮层部分神经元对亮度的反应要慢于对比度反应,被认为是由边界对比度诱导的亮度知觉(brightness)的神经基础.我们的研究表明,初级视皮层许多神经元的亮度反应要快于对比度反应,并且这些神经元偏好低的空间频率、高的时间频率和高的运动速度,提示皮层下具有低空间频率和高运动速度通路的信息输入对产生初级视皮层神经元的亮度反应有贡献.已经知道初级视皮层神经元对空间频率反应的时间过程是从低空间频率到高空间频率,我们发现的早期亮度反应是对极低空间频率的反应,与这一时间过程是一致的,是这一从粗到细的视觉信息加工过程的第一步,揭示了处理最早的粗的视觉信息的神经基础.另外,初级视皮层含有偏好亮度下降和高运动速度的神经元,这群神经元的活动有助于在光照差的环境中检测高速运动的低亮度物体.     

Unveiling functions of the visual cortex using task-specific deep neural networks

The human visual cortex enables visual perception through a cascade of hierarchical computations in cortical regions with distinct functionalities. Here, we introduce an AI-driven approach to discover the functional mapping of the visual cortex. We related human brain responses to scene images measured with functional MRI (fMRI) systematically to a diverse set of deep neural networks (DNNs) optimized to perform different scene perception tasks. We found a structured mapping between DNN tasks and brain regions along the ventral and dorsal visual streams. Low-level visual tasks mapped onto early brain regions, 3-dimensional scene perception tasks mapped onto the dorsal stream, and semantic tasks mapped onto the ventral stream. This mapping was of high fidelity, with more than 60% of the explainable variance in nine key regions being explained. Together, our results provide a novel functional mapping of the human visual cortex and demonstrate the power of the computational approach.     

Our Faces in the Dog's Brain: Functional Imaging Reveals Temporal Cortex Activation during Perception of Human Faces

Dogs have a rich social relationship with humans. One fundamental aspect of it is how dogs pay close attention to human faces in order to guide their behavior, for example, by recognizing their owner and his/her emotional state using visual cues. It is well known that humans have specific brain regions for the processing of other human faces, yet it is unclear how dogs’ brains process human faces. For this reason, our study focuses on describing the brain correlates of perception of human faces in dogs using functional magnetic resonance imaging (fMRI). We trained seven domestic dogs to remain awake, still and unrestrained inside an MRI scanner. We used a visual stimulation paradigm with block design to compare activity elicited by human faces against everyday objects. Brain activity related to the perception of faces changed significantly in several brain regions, but mainly in the bilateral temporal cortex. The opposite contrast (i.e., everyday objects against human faces) showed no significant brain activity change. The temporal cortex is part of the ventral visual pathway, and our results are consistent with reports in other species like primates and sheep, that suggest a high degree of evolutionary conservation of this pathway for face processing. This study introduces the temporal cortex as candidate to process human faces, a pillar of social cognition in dogs.     

Gaze control: role of the parietal cortex

Eye movements constitute one of the most basic means of interacting with our environment, allowing to orient to, localize and scrutinize the variety of potentially interesting objects that surround us. In this review we discuss the role of the parietal cortex in the control of saccadic and smooth pursuit eye movements, whose purpose is to rapidly displace the line of gaze and to maintain a moving object on the central retina, respectively. From single cell recording studies in monkey we know that distinct sub-regions of the parietal lobe are implicated in these two kinds of movement. The middle temporal (MT) and medial superior temporal (MST) areas show neuronal activities related to moving visual stimuli and to ocular pursuit. The lateral intraparietal (LIP) area exhibits visual and saccadic neuronal responses. Electrophysiology, which in essence is a correlation method, cannot entirely solve the question of the functional implication of these areas: are they primarily involved in sensory processing, in motor processing, or in some intermediate function? Lesion approaches (reversible or permanent) in the monkey can provide important information in this respect. Lesions of MT or MST produce deficits in the perception of visual motion, which would argue for their possible role in sensory guidance of ocular pursuit rather than in directing motor commands to the eye muscle. Lesions of LIP do not produce specific visual impairments and cause only subtle saccadic deficits. However, recent results have shown the presence of severe deficits in spatial attention tasks. LIP could thus be implicated in the selection of relevant objects in the visual scene and provide a signal for directing the eyes toward these objects. Functional imaging studies in humans confirm the role of the parietal cortex in pursuit, saccadic, and attentional networks, and show a high degree of overlap with monkey data. Parietal lobe lesions in humans also result in behavioral deficits very similar to those that are observed in the monkey. Altogether, these different sources of data consistently point to the involvement of the parietal cortex in the representation of space, at an intermediate stage between vision and action.     

Neural population representation hypothesis of visual flow and its illusory after effect in the brain: psychophysics,neurophysiology and computational approaches

The neural representation of motion aftereffects induced by various visual flows (translational, rotational, motion-in-depth, and translational transparent flows) was studied under the hypothesis that the imbalances in discharge activities would occur in favor in the direction opposite to the adapting stimulation in the monkey MST cells (cells in the medial superior temporal area) which can discriminate the mode (i.e., translational, rotational, or motion-in-depth) of the given flow. In single-unit recording experiments conducted on anaesthetized monkeys, we found that the rate of spontaneous discharge and the sensitivity to a test stimulus moving in the preferred direction decreased after receiving an adapting stimulation moving in the preferred direction, whereas they increased after receiving an adapting stimulation moving in the null direction. To consistently explain the bidirectional perception of a transparent visual flow and its unidirectional motion aftereffect by the same hypothesis, we need to assume the existence of two subtypes of MST D cells which show directionally selective responses to a translational flow: component cells and integration cells. Our physiological investigation revealed that the MST D cells could be divided into two types: one responded to a transparent flow by two peaks at the instances when the direction of one of the component flow matched the preferred direction of the cell, and the other responded by a single peak at the instance when the direction of the integrated motion matched the preferred direction. In psychophysical experiments on human subjects, we found evidence for the existence of component and integration representations in the human brain. To explain the different motion perceptions, i.e., two transparent flows during presentation of the flows and a single flow in the opposite direction to the integrated flows after stopping the flow stimuli, we suggest that the pattern-discrimination system can select the motion representation that is consistent with the perception of the pattern from two motion representations. We discuss the computational aspects related to the integration of component motion fields.     

纹外皮层PMLS区反馈投射对初级视皮层神经元反应特性的影响

神经系统中存在大量下行投射,与上行输入一起形成复杂的前馈与反馈回路,调控神经信号的传导和处理,但目前对皮层内反馈投射的功能作用认识还比较薄弱.通过微量注射抑制性神经递质γ-氨基丁酸(γ-aminobutyric acid,GABA),使猫纹外皮层后内侧外上雪氏区(area posteromedial lateral suprasylvian,PMLS)局部可逆性失活,使用胞外记录方法,研究初级视皮层17区神经元反应特性的变化.实验结果显示,PMLS区失活后,17区细胞对运动刺激的反应总体减弱,反应的相对稳定性基本不变,最高发放率/自发之比有所下降.与此同时,细胞的方向选择性指数减小,朝向选择性无显著变化.除少数"双向"反应细胞外,绝大部分细胞的最优方向基本不变.进一步分析发现,细胞对各个方向刺激的反应普遍下降,最优方向上的下降程度最大,是导致方向选择性减弱的主要原因.这些结果表明,PMLS区反馈投射可增强初级视皮层的方向选择性,而对朝向选择性影响有限.这一作用特点体现了PMLS区在皮层中偏重处理运动视觉信息的功能.     

The functional neuroanatomy of object agnosia: a case study

Cortical reorganization of visual and object representations following neural injury was examined using fMRI and behavioral investigations. We probed the visual responsivity of the ventral visual cortex of an agnosic patient who was impaired at object recognition following a lesion to the right lateral fusiform gyrus. In both hemispheres, retinotopic mapping revealed typical topographic organization and visual activation of early visual cortex. However, visual responses, object-related, and -selective responses were reduced in regions immediately surrounding the lesion in the right hemisphere, and also, surprisingly, in corresponding locations in the structurally intact left hemisphere. In contrast, hV4 of the right hemisphere showed expanded response properties. These findings indicate that the right lateral fusiform gyrus is critically involved in object recognition and that an impairment to this region has widespread consequences for remote parts of cortex. Finally, functional neural plasticity is possible even when a cortical lesion is sustained in adulthood.     

Mapping the organization of axis of motion selective features in human area MT using high-field fMRI

Functional magnetic resonance imaging (fMRI) at high magnetic fields has made it possible to investigate the columnar organization of the human brain in vivo with high degrees of accuracy and sensitivity. Until now, these results have been limited to the organization principles of early visual cortex (V1). While the middle temporal area (MT) has been the first identified extra-striate visual area shown to exhibit a columnar organization in monkeys, evidence of MT's columnar response properties and topographic layout in humans has remained elusive. Research using various approaches suggests similar response properties as in monkeys but failed to provide direct evidence for direction or axis of motion selectivity in human area MT. By combining state of the art pulse sequence design, high spatial resolution in all three dimensions (0.8 mm isotropic), optimized coil design, ultrahigh field magnets (7 Tesla) and novel high resolution cortical grid sampling analysis tools, we provide the first direct evidence for large-scale axis of motion selective feature organization in human area MT closely matching predictions from topographic columnar-level simulations.     

Dynamic shape integration in extrastriate cortex

BACKGROUND: In anorthoscopic viewing conditions, observers can perceive a moving object through a narrow slit even when only portions of its contour are visible at any time. We used fMRI to examine the contribution of early and later visual cortical areas to dynamic shape integration. Observers' success at integrating the shape of the slit-viewed object was manipulated by varying the degree to which the stimulus was dynamically distorted. Line drawings of common objects were either moderately distorted, strongly distorted, or shown undistorted. Phenomenologically, increasing the stimulus distortion made both object shape and motion more difficult to perceive.RESULTS: We found that bilateral cortical activity in portions of the ventral occipital cortex, corresponding to known object areas within the lateral occipital complex (LOC), was inversely correlated with the degree of stimulus distortion. We found that activity in left MT+, the human cortical area specialized for motion, showed a similar pattern as the ventral occipital region. The LOC also showed greater activity to a fully visible moving object than to the undistorted slit-viewed object. Area MT+, however, showed more equivalent activity to both the slit-viewed and fully visible moving objects.CONCLUSIONS: In early retinotopic cortex, the distorted and undistorted stimuli elicited the same amount of activity. Higher visual areas, however, were correlated with the percept of the coherent object, and this correlation suggests that the shape integration is mediated by later visual cortical areas. Motion information from the dorsal stream may project to the LOC to produce the shape percept.     

Visual stability and the motion aftereffect: a psychophysical study revealing spatial updating

Eye movements create an ever-changing image of the world on the retina. In particular, frequent saccades call for a compensatory mechanism to transform the changing visual information into a stable percept. To this end, the brain presumably uses internal copies of motor commands. Electrophysiological recordings of visual neurons in the primate lateral intraparietal cortex, the frontal eye fields, and the superior colliculus suggest that the receptive fields (RFs) of special neurons shift towards their post-saccadic positions before the onset of a saccade. However, the perceptual consequences of these shifts remain controversial. We wanted to test in humans whether a remapping of motion adaptation occurs in visual perception.The motion aftereffect (MAE) occurs after viewing of a moving stimulus as an apparent movement to the opposite direction. We designed a saccade paradigm suitable for revealing pre-saccadic remapping of the MAE. Indeed, a transfer of motion adaptation from pre-saccadic to post-saccadic position could be observed when subjects prepared saccades. In the remapping condition, the strength of the MAE was comparable to the effect measured in a control condition (33±7% vs. 27±4%). Contrary, after a saccade or without saccade planning, the MAE was weak or absent when adaptation and test stimulus were located at different retinal locations, i.e. the effect was clearly retinotopic. Regarding visual cognition, our study reveals for the first time predictive remapping of the MAE but no spatiotopic transfer across saccades. Since the cortical sites involved in motion adaptation in primates are most likely the primary visual cortex and the middle temporal area (MT/V5) corresponding to human MT, our results suggest that pre-saccadic remapping extends to these areas, which have been associated with strict retinotopy and therefore with classical RF organization. The pre-saccadic transfer of visual features demonstrated here may be a crucial determinant for a stable percept despite saccades.     

Primary visual cortex activity along the apparent-motion trace reflects illusory perception

The illusion of apparent motion can be induced when visual stimuli are successively presented at different locations. It has been shown in previous studies that motion-sensitive regions in extrastriate cortex are relevant for the processing of apparent motion, but it is unclear whether primary visual cortex (V1) is also involved in the representation of the illusory motion path. We investigated, in human subjects, apparent-motion-related activity in patches of V1 representing locations along the path of illusory stimulus motion using functional magnetic resonance imaging. Here we show that apparent motion caused a blood-oxygenation-level-dependent response along the V1 representations of the apparent-motion path, including regions that were not directly activated by the apparent-motion-inducing stimuli. This response was unaltered when participants had to perform an attention-demanding task that diverted their attention away from the stimulus. With a bistable motion quartet, we confirmed that the activity was related to the conscious perception of movement. Our data suggest that V1 is part of the network that represents the illusory path of apparent motion. The activation in V1 can be explained either by lateral interactions within V1 or by feedback mechanisms from higher visual areas, especially the motion-sensitive human MT/V5 complex.