Distributed neural plasticity for shape learning in the human visual cortex

Expertise in recognizing objects in cluttered scenes is a critical skill for our interactions in complex environments and is thought to develop with learning. However, the neural implementation of object learning across stages of visual analysis in the human brain remains largely unknown. Using combined psychophysics and functional magnetic resonance imaging (fMRI), we show a link between shape-specific learning in cluttered scenes and distributed neuronal plasticity in the human visual cortex. We report stronger fMRI responses for trained than untrained shapes across early and higher visual areas when observers learned to detect low-salience shapes in noisy backgrounds. However, training with high-salience pop-out targets resulted in lower fMRI responses for trained than untrained shapes in higher occipitotemporal areas. These findings suggest that learning of camouflaged shapes is mediated by increasing neural sensitivity across visual areas to bolster target segmentation and feature integration. In contrast, learning of prominent pop-out shapes is mediated by associations at higher occipitotemporal areas that support sparser coding of the critical features for target recognition. We propose that the human brain learns novel objects in complex scenes by reorganizing shape processing across visual areas, while taking advantage of natural image correlations that determine the distinctiveness of target shapes.     

Multivoxel Patterns in Fusiform Face Area Differentiate Faces by Sex and Race

Although prior research suggests that fusiform gyrus represents the sex and race of faces, it remains unclear whether fusiform face area (FFA)–the portion of fusiform gyrus that is functionally-defined by its preferential response to faces–contains such representations. Here, we used functional magnetic resonance imaging to evaluate whether FFA represents faces by sex and race. Participants were scanned while they categorized the sex and race of unfamiliar Black men, Black women, White men, and White women. Multivariate pattern analysis revealed that multivoxel patterns in FFA–but not other face-selective brain regions, other category-selective brain regions, or early visual cortex–differentiated faces by sex and race. Specifically, patterns of voxel-based responses were more similar between individuals of the same sex than between men and women, and between individuals of the same race than between Black and White individuals. By showing that FFA represents the sex and race of faces, this research contributes to our emerging understanding of how the human brain perceives individuals from two fundamental social categories.     

Perceptual organization of local elements into global shapes in the human visual cortex

The question of how local image features on the retina are integrated into perceived global shapes is central to our understanding of human visual perception. Psychophysical investigations have suggested that the emergence of a coherent visual percept, or a "good-Gestalt", is mediated by the perceptual organization of local features based on their similarity. However, the neural mechanisms that mediate unified shape perception in the human brain remain largely unknown. Using human fMRI, we demonstrate that not only higher occipitotemporal but also early retinotopic areas are involved in the perceptual organization and detection of global shapes. Specifically, these areas showed stronger fMRI responses to global contours consisting of collinear elements than to patterns of randomly oriented local elements. More importantly, decreased detection performance and fMRI activations were observed when misalignment of the contour elements disturbed the perceptual coherence of the contours. However, grouping of the misaligned contour elements by disparity resulted in increased performance and fMRI activations, suggesting that similar neural mechanisms may underlie grouping of local elements to global shapes by different visual features (orientation or disparity). Thus, these findings provide novel evidence for the role of both early feature integration processes and higher stages of visual analysis in coherent visual perception.     

Preference for Orientations Commonly Viewed for One’s Own Hand in the Anterior Intraparietal Cortex

Brain regions in the intraparietal and the premotor cortices selectively process visual and multisensory events near the hands (peri-hand space). Visual information from the hand itself modulates this processing potentially because it is used to estimate the location of one’s own body and the surrounding space. In humans specific occipitotemporal areas process visual information of specific body parts such as hands. Here we used an fMRI block-design to investigate if anterior intraparietal and ventral premotor ‘peri-hand areas’ exhibit selective responses to viewing images of hands and viewing specific hand orientations. Furthermore, we investigated if the occipitotemporal ‘hand area’ is sensitive to viewed hand orientation. Our findings demonstrate increased BOLD responses in the left anterior intraparietal area when participants viewed hands and feet as compared to faces and objects. Anterior intraparietal and also occipitotemporal areas in the left hemisphere exhibited response preferences for viewing right hands with orientations commonly viewed for one’s own hand as compared to uncommon own hand orientations. Our results indicate that both anterior intraparietal and occipitotemporal areas encode visual limb-specific shape and orientation information.     

Early category-specific cortical activation revealed by visual stimulus inversion

Visual categorization may already start within the first 100-ms after stimulus onset, in contrast with the long-held view that during this early stage all complex stimuli are processed equally and that category-specific cortical activation occurs only at later stages. The neural basis of this proposed early stage of high-level analysis is however poorly understood. To address this question we used magnetoencephalography and anatomically-constrained distributed source modeling to monitor brain activity with millisecond-resolution while subjects performed an orientation task on the upright and upside-down presented images of three different stimulus categories: faces, houses and bodies. Significant inversion effects were found for all three stimulus categories between 70-100-ms after picture onset with a highly category-specific cortical distribution. Differential responses between upright and inverted faces were found in well-established face-selective areas of the inferior occipital cortex and right fusiform gyrus. In addition, early category-specific inversion effects were found well beyond visual areas. Our results provide the first direct evidence that category-specific processing in high-level category-sensitive cortical areas already takes place within the first 100-ms of visual processing, significantly earlier than previously thought, and suggests the existence of fast category-specific neocortical routes in the human brain.     

The Neural Correlates of Visuospatial Perceptual and Oculomotor Extrapolation

The human visual system must perform complex visuospatial extrapolations (VSE) across space and time in order to extract shape and form from the retinal projection of a cluttered visual environment characterized by occluded surfaces and moving objects. Even if we exclude the temporal dimension, for instance when judging whether an extended finger is pointing towards one object or another, the mechanisms of VSE remain opaque. Here we investigated the neural correlates of VSE using functional magnetic resonance imaging in sixteen human observers while they judged the relative position of, or saccaded to, a (virtual) target defined by the extrapolated path of a pointer. Using whole brain and region of interest (ROI) analyses, we compared the brain activity evoked by these VSE tasks to similar control judgements or eye movements made to explicit (dot) targets that did not require extrapolation. The data show that activity in an occipitotemporal region that included the lateral occipital cortex (LOC) was significantly greater during VSE than during control tasks. A similar, though less pronounced, pattern was also evident in regions of the fronto-parietal cortex that included the frontal eye fields. However, none of the ROIs examined exhibited a significant interaction between target type (extrapolated/explicit) and response type (oculomotor/perceptual). These findings are consistent with a close association between visuoperceptual and oculomotor responses, and highlight a critical role for the LOC in the process of VSE.     

Hierarchical coding of letter strings in the ventral stream: dissecting the inner organization of the visual word-form system

Visual word recognition has been proposed to rely on a hierarchy of increasingly complex neuronal detectors, from individual letters to bigrams and morphemes. We used fMRI to test whether such a hierarchy is present in the left occipitotemporal cortex, at the site of the visual word-form area, and with an anterior-to-posterior progression. We exposed adult readers to (1) false-font strings; (2) strings of infrequent letters; (3) strings of frequent letters but rare bigrams; (4) strings with frequent bigrams but rare quadrigrams; (5) strings with frequent quadrigrams; (6) real words. A gradient of selectivity was observed through the entire span of the occipitotemporal cortex, with activation becoming more selective for higher-level stimuli toward the anterior fusiform region. A similar gradient was also seen in left inferior frontoinsular cortex. Those gradients were asymmetrical in favor of the left hemisphere. We conclude that the left occipitotemporal visual word-form area, far from being a homogeneous structure, presents a high degree of functional and spatial hierarchical organization which must result from a tuning process during reading acquisition.     

In the eye of the beholder: visual experience and categories in the human brain

How does experience change representations of visual objects in the brain? Do cortical object representations reflect category membership? In this issue of Neuron, Jiang et al. show that category training leads to sharpening of neural responses in high-level visual cortex; in contrast, category boundaries may be represented only in prefrontal cortex.     

Distinct cerebral pathways for object identity and number in human infants

All humans, regardless of their culture and education, possess an intuitive understanding of number. Behavioural evidence suggests that numerical competence may be present early on in infancy. Here, we present brain-imaging evidence for distinct cerebral coding of number and object identity in 3-mo-old infants. We compared the visual event-related potentials evoked by unforeseen changes either in the identity of objects forming a set, or in the cardinal of this set. In adults and 4-y-old children, number sense relies on a dorsal system of bilateral intraparietal areas, different from the ventral occipitotemporal system sensitive to object identity. Scalp voltage topographies and cortical source modelling revealed a similar distinction in 3-mo-olds, with changes in object identity activating ventral temporal areas, whereas changes in number involved an additional right parietoprefrontal network. These results underscore the developmental continuity of number sense by pointing to early functional biases in brain organization that may channel subsequent learning to restricted brain areas.     

Magnetic stimulation of extrastriate body area impairs visual processing of nonfacial body parts

Functional magnetic resonance imaging indicates that observation of the human body induces a selective activation of a lateral occipitotemporal cortical area called extrastriate body area (EBA). This area is responsive to static and moving images of the human body and parts of it, but it is insensitive to faces and stimulus categories unrelated to the human body. With event-related repetitive transcranial magnetic stimulation, we tested the possible causal relation between neural activity in EBA and visual processing of body-related, nonfacial stimuli. Facial and noncorporeal stimuli were used as a control. Interference with neural activity in EBA induced a clear impairment, consisting of a significant increase in discriminative reaction time, in the visual processing of body parts. The effect was selective for stimulus type, because it affected responses to nonfacial body stimuli but not to noncorporeal and facial stimuli, and for locus of stimulation, because the effect from the interfering stimulation of EBA was absent during a corresponding stimulation of primary visual cortex. The results provide strong evidence that neural activity in EBA is not only correlated with but also causally involved in the visual processing of the human body and its parts, except the face.     

Cerebral Asymmetry of fMRI-BOLD Responses to Visual Stimulation

Hemispheric asymmetry of a wide range of functions is a hallmark of the human brain. The visual system has traditionally been thought of as symmetrically distributed in the brain, but a growing body of evidence has challenged this view. Some highly specific visual tasks have been shown to depend on hemispheric specialization. However, the possible lateralization of cerebral responses to a simple checkerboard visual stimulation has not been a focus of previous studies. To investigate this, we performed two sessions of blood-oxygenation level dependent (BOLD) functional magnetic resonance imaging (fMRI) in 54 healthy subjects during stimulation with a black and white checkerboard visual stimulus. While carefully excluding possible non-physiological causes of left-to-right bias, we compared the activation of the left and the right cerebral hemispheres and related this to grey matter volume, handedness, age, gender, ocular dominance, interocular difference in visual acuity, as well as line-bisection performance. We found a general lateralization of cerebral activation towards the right hemisphere of early visual cortical areas and areas of higher-level visual processing, involved in visuospatial attention, especially in top-down (i.e., goal-oriented) attentional processing. This right hemisphere lateralization was partly, but not completely, explained by an increased grey matter volume in the right hemisphere of the early visual areas. Difference in activation of the superior parietal lobule was correlated with subject age, suggesting a shift towards the left hemisphere with increasing age. Our findings suggest a right-hemispheric dominance of these areas, which could lend support to the generally observed leftward visual attentional bias and to the left hemifield advantage for some visual perception tasks.     

A Representational Similarity Analysis of the Dynamics of Object Processing Using Single-Trial EEG Classification

The recognition of object categories is effortlessly accomplished in everyday life, yet its neural underpinnings remain not fully understood. In this electroencephalography (EEG) study, we used single-trial classification to perform a Representational Similarity Analysis (RSA) of categorical representation of objects in human visual cortex. Brain responses were recorded while participants viewed a set of 72 photographs of objects with a planned category structure. The Representational Dissimilarity Matrix (RDM) used for RSA was derived from confusions of a linear classifier operating on single EEG trials. In contrast to past studies, which used pairwise correlation or classification to derive the RDM, we used confusion matrices from multi-class classifications, which provided novel self-similarity measures that were used to derive the overall size of the representational space. We additionally performed classifications on subsets of the brain response in order to identify spatial and temporal EEG components that best discriminated object categories and exemplars. Results from category-level classifications revealed that brain responses to images of human faces formed the most distinct category, while responses to images from the two inanimate categories formed a single category cluster. Exemplar-level classifications produced a broadly similar category structure, as well as sub-clusters corresponding to natural language categories. Spatiotemporal components of the brain response that differentiated exemplars within a category were found to differ from those implicated in differentiating between categories. Our results show that a classification approach can be successfully applied to single-trial scalp-recorded EEG to recover fine-grained object category structure, as well as to identify interpretable spatiotemporal components underlying object processing. Finally, object category can be decoded from purely temporal information recorded at single electrodes.     

The functional neuroanatomy of object agnosia: a case study

Cortical reorganization of visual and object representations following neural injury was examined using fMRI and behavioral investigations. We probed the visual responsivity of the ventral visual cortex of an agnosic patient who was impaired at object recognition following a lesion to the right lateral fusiform gyrus. In both hemispheres, retinotopic mapping revealed typical topographic organization and visual activation of early visual cortex. However, visual responses, object-related, and -selective responses were reduced in regions immediately surrounding the lesion in the right hemisphere, and also, surprisingly, in corresponding locations in the structurally intact left hemisphere. In contrast, hV4 of the right hemisphere showed expanded response properties. These findings indicate that the right lateral fusiform gyrus is critically involved in object recognition and that an impairment to this region has widespread consequences for remote parts of cortex. Finally, functional neural plasticity is possible even when a cortical lesion is sustained in adulthood.     

Separate processing of different global-motion structures in visual cortex is revealed by FMRI

The visual system has the remarkable ability to extract several types of meaningful global-motion signals, such as radial motion, translation motion, and rotation, for different visual functions and actions. In the monkey brain, different groups of cells in MST respond best to different types of global motion [1, 2] whereas in lower cortical areas including MT, no such differential responses have been found. Here, we show that an area (or areas) lower than MST in the human brain [3] responds to different types of global motion. A series of human functional magnetic resonance imaging (fMRI) experiments, in which attention was controlled for, indicated that the center of radial motion activates the corresponding location in the V3A representation, whereas translation motion activates mainly in a more peripheral representation of V3A. These results suggest that in the human brain, V3A is an area that differentially responds according to the type of global motion.     

Reconstructing visual experiences from brain activity evoked by natural movies

Quantitative modeling of human brain activity can provide crucial insights about cortical representations [1, 2] and can form the basis for brain decoding devices [3-5]. Recent functional magnetic resonance imaging (fMRI) studies have modeled brain activity elicited by static visual patterns and have reconstructed these patterns from brain activity [6-8]. However, blood oxygen level-dependent (BOLD) signals measured via fMRI are very slow [9], so it has been difficult to model brain activity elicited by dynamic stimuli such as natural movies. Here we present a new motion-energy [10, 11] encoding model that largely overcomes this limitation. The model describes fast visual information and slow hemodynamics by separate components. We recorded BOLD signals in occipitotemporal visual cortex of human subjects who watched natural movies and fit the model separately to individual voxels. Visualization of the fit models reveals how early visual areas represent the information in movies. To demonstrate the power of our approach, we also constructed a Bayesian decoder [8] by combining estimated encoding models with a sampled natural movie prior. The decoder provides remarkable reconstructions of the viewed movies. These results demonstrate that dynamic brain activity measured under naturalistic conditions can be decoded using current fMRI technology.     

The naked truth: the face and body sensitive N170 response is enhanced for nude bodies

Recent event-related potential studies have shown that the occipitotemporal N170 component--best known for its sensitivity to faces--is also sensitive to perception of human bodies. Considering that in the timescale of evolution clothing is a relatively new invention that hides the bodily features relevant for sexual selection and arousal, we investigated whether the early N170 brain response would be enhanced to nude over clothed bodies. In two experiments, we measured N170 responses to nude bodies, bodies wearing swimsuits, clothed bodies, faces, and control stimuli (cars). We found that the N170 amplitude was larger to opposite and same-sex nude vs. clothed bodies. Moreover, the N170 amplitude increased linearly as the amount of clothing decreased from full clothing via swimsuits to nude bodies. Strikingly, the N170 response to nude bodies was even greater than that to faces, and the N170 amplitude to bodies was independent of whether the face of the bodies was visible or not. All human stimuli evoked greater N170 responses than did the control stimulus. Autonomic measurements and self-evaluations showed that nude bodies were affectively more arousing compared to the other stimulus categories. We conclude that the early visual processing of human bodies is sensitive to the visibility of the sex-related features of human bodies and that the visual processing of other people's nude bodies is enhanced in the brain. This enhancement is likely to reflect affective arousal elicited by nude bodies. Such facilitated visual processing of other people's nude bodies is possibly beneficial in identifying potential mating partners and competitors, and for triggering sexual behavior.     

Concurrent TMS-fMRI and psychophysics reveal frontal influences on human retinotopic visual cortex

BACKGROUND: Regions in human frontal cortex may have modulatory top-down influences on retinotopic visual cortex, but to date neuroimaging methods have only been able to provide indirect evidence for such functional interactions between remote but interconnected brain regions. Here we combined transcranial magnetic stimulation (TMS) with concurrent functional magnetic resonance imaging (fMRI), plus psychophysics, to show that stimulation of the right human frontal eye-field (FEF) produced a characteristic topographic pattern of activity changes in retinotopic visual areas V1-V4, with functional consequences for visual perception. RESULTS: FEF TMS led to activity increases for retinotopic representations of the peripheral visual field, but to activity decreases for the central field, in areas V1-V4. These frontal influences on visual cortex occurred in a top-down manner, independently of visual input. TMS of a control site (vertex) did not elicit such visual modulations, and saccades, blinks, or pupil dilation could not account for our results. Finally, the effects of FEF TMS on activity in retinotopic visual cortex led to a behavioral prediction that we confirmed psychophysically by showing that TMS of the frontal site (again compared with vertex) enhanced perceived contrast for peripheral relative to central visual stimuli. CONCLUSIONS: Our results provide causal evidence that circuits originating in the human FEF can modulate activity in retinotopic visual cortex, in a manner that differentiates the central and peripheral visual field, with functional consequences for perception. More generally, our study illustrates how the new approach of concurrent TMS-fMRI can now reveal causal interactions between remote but interconnected areas of the human brain.     

Beyond Shape: How You Learn about Objects Affects How They Are Represented in Visual Cortex

Background

Experience can alter how objects are represented in the visual cortex. But experience can take different forms. It is unknown whether the kind of visual experience systematically alters the nature of visual cortical object representations.

Methodology/Principal Findings

We take advantage of different training regimens found to produce qualitatively different types of perceptual expertise behaviorally in order to contrast the neural changes that follow different kinds of visual experience with the same objects. Two groups of participants went through training regimens that required either subordinate-level individuation or basic-level categorization of a set of novel, artificial objects, called “Ziggerins”. fMRI activity of a region in the right fusiform gyrus increased after individuation training and was correlated with the magnitude of configural processing of the Ziggerins observed behaviorally. In contrast, categorization training caused distributed changes, with increased activity in the medial portion of the ventral occipito-temporal cortex relative to more lateral areas.

Conclusions/Significance

Our results demonstrate that the kind of experience with a category of objects can systematically influence how those objects are represented in visual cortex. The demands of prior learning experience therefore appear to be one factor determining the organization of activity patterns in visual cortex.