Beyond Motor Scheme: A Supramodal Distributed Representation in the Action-Observation Network

The representation of actions within the action-observation network is thought to rely on a distributed functional organization. Furthermore, recent findings indicate that the action-observation network encodes not merely the observed motor act, but rather a representation that is independent from a specific sensory modality or sensory experience. In the present study, we wished to determine to what extent this distributed and ‘more abstract’ representation of action is truly supramodal, i.e. shares a common coding across sensory modalities. To this aim, a pattern recognition approach was employed to analyze neural responses in sighted and congenitally blind subjects during visual and/or auditory presentation of hand-made actions. Multivoxel pattern analyses-based classifiers discriminated action from non-action stimuli across sensory conditions (visual and auditory) and experimental groups (blind and sighted). Moreover, these classifiers labeled as ‘action’ the pattern of neural responses evoked during actual motor execution. Interestingly, discriminative information for the action/non action classification was located in a bilateral, but left-prevalent, network that strongly overlaps with brain regions known to form the action-observation network and the human mirror system. The ability to identify action features with a multivoxel pattern analyses-based classifier in both sighted and blind individuals and independently from the sensory modality conveying the stimuli clearly supports the hypothesis of a supramodal, distributed functional representation of actions, mainly within the action-observation network.     

Preserved functional specialization for spatial processing in the middle occipital gyrus of the early blind

The occipital cortex (OC) of early-blind humans is activated during various nonvisual perceptual and cognitive tasks, but little is known about its modular organization. Using functional MRI we tested whether processing of auditory versus tactile and spatial versus nonspatial information was dissociated in the OC of the early blind. No modality-specific OC activation was observed. However, the right middle occipital gyrus (MOG) showed a preference for spatial over nonspatial processing of both auditory and tactile stimuli. Furthermore, MOG activity was correlated with accuracy of individual sound localization performance. In sighted controls, most of extrastriate OC, including the MOG, was deactivated during auditory and tactile conditions, but the right MOG was more activated during spatial than nonspatial visual tasks. Thus, although the sensory modalities driving the neurons in the reorganized OC of blind individuals are altered, the functional specialization of extrastriate cortex is retained regardless of visual experience.     

Modality-specific control of strategic spatial attention in parietal cortex

The neural basis of selective spatial attention presents a significant challenge to cognitive neuroscience. Recent neuroimaging studies have suggested that regions of the parietal and temporal cortex constitute a "supramodal" network that mediates goal-directed attention in multiple sensory modalities. Here we used transcranial magnetic stimulation (TMS) to determine which cortical subregions control strategic attention in vision and touch. Healthy observers undertook an orienting task in which a central arrow cue predicted the location of a subsequent visual or somatosensory target. To determine the attentional role of cortical subregions at different stages of processing, TMS was delivered to the right hemisphere during cue or target events. Results indicated a critical role of the inferior parietal cortex in strategic orienting to visual events, but not to somatosensory events. These findings are inconsistent with the existence of a supramodal attentional network and instead provide direct evidence for modality-specific attentional processing in parietal cortex.     

Right Occipital Cortex Activation Correlates with Superior Odor Processing Performance in the Early Blind

Using functional magnetic resonance imaging (fMRI) in ten early blind humans, we found robust occipital activation during two odor-processing tasks (discrimination or categorization of fruit and flower odors), as well as during control auditory-verbal conditions (discrimination or categorization of fruit and flower names). We also found evidence for reorganization and specialization of the ventral part of the occipital cortex, with dissociation according to stimulus modality: the right fusiform gyrus was most activated during olfactory conditions while part of the left ventral lateral occipital complex showed a preference for auditory-verbal processing. Only little occipital activation was found in sighted subjects, but the same right-olfactory/left-auditory-verbal hemispheric lateralization was found overall in their brain. This difference between the groups was mirrored by superior performance of the blind in various odor-processing tasks. Moreover, the level of right fusiform gyrus activation during the olfactory conditions was highly correlated with individual scores in a variety of odor recognition tests, indicating that the additional occipital activation may play a functional role in odor processing.     

Patterns of Individual Variation in Visual Pathway Structure and Function in the Sighted and Blind

Many structural and functional brain alterations accompany blindness, with substantial individual variation in these effects. In normally sighted people, there is correlated individual variation in some visual pathway structures. Here we examined if the changes in brain anatomy produced by blindness alter the patterns of anatomical variation found in the sighted. We derived eight measures of central visual pathway anatomy from a structural image of the brain from 59 sighted and 53 blind people. These measures showed highly significant differences in mean size between the sighted and blind cohorts. When we examined the measurements across individuals within each group we found three clusters of correlated variation, with V1 surface area and pericalcarine volume linked, and independent of the thickness of V1 cortex. These two clusters were in turn relatively independent of the volumes of the optic chiasm and lateral geniculate nucleus. This same pattern of variation in visual pathway anatomy was found in the sighted and the blind. Anatomical changes within these clusters were graded by the timing of onset of blindness, with those subjects with a post-natal onset of blindness having alterations in brain anatomy that were intermediate to those seen in the sighted and congenitally blind. Many of the blind and sighted subjects also contributed functional MRI measures of cross-modal responses within visual cortex, and a diffusion tensor imaging measure of fractional anisotropy within the optic radiations and the splenium of the corpus callosum. We again found group differences between the blind and sighted in these measures. The previously identified clusters of anatomical variation were also found to be differentially related to these additional measures: across subjects, V1 cortical thickness was related to cross-modal activation, and the volume of the optic chiasm and lateral geniculate was related to fractional anisotropy in the visual pathway. Our findings show that several of the structural and functional effects of blindness may be reduced to a smaller set of dimensions. It also seems that the changes in the brain that accompany blindness are on a continuum with normal variation found in the sighted.     

Rapid and reversible recruitment of early visual cortex for touch

Background

The loss of vision has been associated with enhanced performance in non-visual tasks such as tactile discrimination and sound localization. Current evidence suggests that these functional gains are linked to the recruitment of the occipital visual cortex for non-visual processing, but the neurophysiological mechanisms underlying these crossmodal changes remain uncertain. One possible explanation is that visual deprivation is associated with an unmasking of non-visual input into visual cortex.

Methodology/Principal Findings

We investigated the effect of sudden, complete and prolonged visual deprivation (five days) in normally sighted adult individuals while they were immersed in an intensive tactile training program. Following the five-day period, blindfolded subjects performed better on a Braille character discrimination task. In the blindfold group, serial fMRI scans revealed an increase in BOLD signal within the occipital cortex in response to tactile stimulation after five days of complete visual deprivation. This increase in signal was no longer present 24 hours after blindfold removal. Finally, reversible disruption of occipital cortex function on the fifth day (by repetitive transcranial magnetic stimulation; rTMS) impaired Braille character recognition ability in the blindfold group but not in non-blindfolded controls. This disruptive effect was no longer evident once the blindfold had been removed for 24 hours.

Conclusions/Significance

Overall, our findings suggest that sudden and complete visual deprivation in normally sighted individuals can lead to profound, but rapidly reversible, neuroplastic changes by which the occipital cortex becomes engaged in processing of non-visual information. The speed and dynamic nature of the observed changes suggests that normally inhibited or masked functions in the sighted are revealed by visual loss. The unmasking of pre-existing connections and shifts in connectivity represent rapid, early plastic changes, which presumably can lead, if sustained and reinforced, to slower developing, but more permanent structural changes, such as the establishment of new neural connections in the blind.     

Auditory Attention Activates Peripheral Visual Cortex

Background

Recent neuroimaging studies have revealed that putatively unimodal regions of visual cortex can be activated during auditory tasks in sighted as well as in blind subjects. However, the task determinants and functional significance of auditory occipital activations (AOAs) remains unclear.

Methodology/Principal Findings

We examined AOAs in an intermodal selective attention task to distinguish whether they were stimulus-bound or recruited by higher-level cognitive operations associated with auditory attention. Cortical surface mapping showed that auditory occipital activations were localized to retinotopic visual cortex subserving the far peripheral visual field. AOAs depended strictly on the sustained engagement of auditory attention and were enhanced in more difficult listening conditions. In contrast, unattended sounds produced no AOAs regardless of their intensity, spatial location, or frequency.

Conclusions/Significance

Auditory attention, but not passive exposure to sounds, routinely activated peripheral regions of visual cortex when subjects attended to sound sources outside the visual field. Functional connections between auditory cortex and visual cortex subserving the peripheral visual field appear to underlie the generation of AOAs, which may reflect the priming of visual regions to process soon-to-appear objects associated with unseen sound sources.     

Navigation Using Sensory Substitution in Real and Virtual Mazes

Under certain specific conditions people who are blind have a perception of space that is equivalent to that of sighted individuals. However, in most cases their spatial perception is impaired. Is this simply due to their current lack of access to visual information or does the lack of visual information throughout development prevent the proper integration of the neural systems underlying spatial cognition? Sensory Substitution devices (SSDs) can transfer visual information via other senses and provide a unique tool to examine this question. We hypothesize that the use of our SSD (The EyeCane: a device that translates distance information into sounds and vibrations) can enable blind people to attain a similar performance level as the sighted in a spatial navigation task. We gave fifty-six participants training with the EyeCane. They navigated in real life-size mazes using the EyeCane SSD and in virtual renditions of the same mazes using a virtual-EyeCane. The participants were divided into four groups according to visual experience: congenitally blind, low vision & late blind, blindfolded sighted and sighted visual controls. We found that with the EyeCane participants made fewer errors in the maze, had fewer collisions, and completed the maze in less time on the last session compared to the first. By the third session, participants improved to the point where individual trials were no longer significantly different from the initial performance of the sighted visual group in terms of errors, time and collision.     

Behavioral and neuroplastic changes in the blind: evidence for functionally relevant cross-modal interactions.

The study of blind individuals provides insight into the brain re-organization and behavioral compensations that occur following sensory deprivation. While behavioral studies have yielded conflicting results in terms of performance levels within the remaining senses, deafferentation of visual cortical areas through peripheral blindness results in clear neuroplastic changes. Most striking is the activation of occipital cortex in response to auditory and tactile stimulation. Indeed, parts of the "unimodal" visual cortex are recruited by other sensory modalities to process sensory information in a functionally relevant manner. In addition, a larger area of the sensorimotor cortex is devoted to the representation of the reading finger in blind Braille readers. The "visual" function of the deafferented occipital cortex is also altered, where transcranial magnetic stimulation-induced phosphenes can be elicited in only 20% of blind subjects. The neural mechanisms underlying these changes remain elusive but recent data showing rapid cross-modal plasticity in blindfolded, sighted subjects argue against the establishment of new connections to explain cross-modal interactions in the blind. Rather, latent pathways that participate in multisensory percepts in sighted subjects might be unmasked and may be potentiated in the event of complete loss of visual input. These issues have important implications for the development of visual prosthesis aimed at restoring some degree of vision in the blind.     

Numbers in the Blind's “Eye”

Background

Although lacking visual experience with numerosities, recent evidence shows that the blind perform similarly to sighted persons on numerical comparison or parity judgement tasks. In particular, on tasks presented in the auditory modality, the blind surprisingly show the same effect that appears in sighted persons, demonstrating that numbers are represented through a spatial code, i.e. the Spatial-Numerical Association of Response Codes (SNARC) effect. But, if this is the case, how is this numerical spatial representation processed in the brain of the blind?

Principal Findings

Here we report that, although blind and sighted people have similarly organized numerical representations, the attentional shifts generated by numbers have different electrophysiological correlates (sensorial N100 in the sighted and cognitive P300 in the blind).

Conclusions

These results highlight possible differences in the use of spatial representations acquired through modalities other than vision in the blind population.     

A supramodal number representation in human intraparietal cortex

The triple-code theory of numerical processing postulates an abstract-semantic "number sense." Neuropsychology points to intraparietal cortex as a potential substrate, but previous functional neuroimaging studies did not dissociate the representation of numerical magnitude from task-driven effects on intraparietal activation. In an event-related fMRI study, we presented numbers, letters, and colors in the visual and auditory modality, asking subjects to respond to target items within each category. In the absence of explicit magnitude processing, numbers compared with letters and colors across modalities activated a bilateral region in the horizontal intraparietal sulcus. This stimulus-driven number-specific intraparietal response supports the idea of a supramodal number representation that is automatically accessed by presentation of numbers and may code magnitude information.     

Multivoxel Pattern Analysis Reveals Auditory Motion Information in MT+ of Both Congenitally Blind and Sighted Individuals

Cross-modal plasticity refers to the recruitment of cortical regions involved in the processing of one modality (e.g. vision) for processing other modalities (e.g. audition). The principles determining how and where cross-modal plasticity occurs remain poorly understood. Here, we investigate these principles by testing responses to auditory motion in visual motion area MT+ of congenitally blind and sighted individuals. Replicating previous reports, we find that MT+ as a whole shows a strong and selective responses to auditory motion in congenitally blind but not sighted individuals, suggesting that the emergence of this univariate response depends on experience. Importantly, however, multivoxel pattern analyses showed that MT+ contained information about different auditory motion conditions in both blind and sighted individuals. These results were specific to MT+ and not found in early visual cortex. Basic sensitivity to auditory motion in MT+ is thus experience-independent, which may be a basis for the region''s strong cross-modal recruitment in congenital blindness.     

Exploring the Relationship between Semantics and Space

The asymmetric distribution of human spatial attention has been repeatedly documented in both patients and healthy controls. Biases in the distribution of attention and/or in the mental representation of space may also affect some aspects of language processing. We investigated whether biases in attention and/or mental representation of space affect semantic representations. In particular, we investigated whether semantic judgments could be modulated by the location in space where the semantic information was presented and the role of the left and right parietal cortices in this task. Healthy subjects were presented with three pictures arranged horizontally (one middle and two outer pictures) of items belonging to the same semantic category. Subjects were asked to indicate the spatial position in which the semantic distance between the outer and middle pictures was smaller. Subjects systematically overestimated the semantic distance of items presented in the right side of space. We explored the neural correlates underpinning this bias using rTMS over the left and right parietal cortex. rTMS of the left parietal cortex selectively reduced this rightward bias. Our findings suggest the existence of an attentional and/or mental representational bias in semantic judgments, similar to that observed for the processing of space and numbers. Spatial manipulation of semantic material results in the activation of specialised attentional resources located in the left hemisphere.     

Modality-independent coding of spatial layout in the human brain

In many nonhuman species, neural computations of navigational information such as position and orientation are not tied to a specific sensory modality [1, 2]. Rather, spatial signals are integrated from multiple input sources, likely leading to abstract representations of space. In contrast, the potential for abstract spatial representations in humans is not known, because most neuroscientific experiments on human navigation have focused exclusively on visual cues. Here, we tested the modality independence hypothesis with two functional magnetic resonance imaging (fMRI) experiments that characterized computations in regions implicated in processing spatial layout [3]. According to the hypothesis, such regions should be recruited for spatial computation of 3D geometric configuration, independent of a specific sensory modality. In support of this view, sighted participants showed strong activation of the parahippocampal place area (PPA) and the retrosplenial cortex (RSC) for visual and haptic exploration of information-matched scenes but not objects. Functional connectivity analyses suggested that these effects were not related to visual recoding, which was further supported by a similar preference for haptic scenes found with blind participants. Taken together, these findings establish the PPA/RSC network as critical in modality-independent spatial computations and provide important evidence for a theory of high-level abstract spatial information processing in the human brain.     

An fMRI Study of Grammatical Morpheme Processing Associated with Nouns and Verbs in Chinese

This study examined whether the degree of complexity of a grammatical component in a language would impact on its representation in the brain through identifying the neural correlates of grammatical morpheme processing associated with nouns and verbs in Chinese. In particular, the processing of Chinese nominal classifiers and verbal aspect markers were investigated in a sentence completion task and a grammaticality judgment task to look for converging evidence. The Chinese language constitutes a special case because it has no inflectional morphology per se and a larger classifier than aspect marker inventory, contrary to the pattern of greater verbal than nominal paradigmatic complexity in most European languages. The functional imaging results showed BA47 and left supplementary motor area and superior medial frontal gyrus more strongly activated for classifier processing, and the left posterior middle temporal gyrus more responsive to aspect marker processing. We attributed the activation in the left prefrontal cortex to greater processing complexity during classifier selection, analogous to the accounts put forth for European languages, and the left posterior middle temporal gyrus to more demanding verb semantic processing. The overall findings significantly contribute to cross-linguistic observations of neural substrates underlying processing of grammatical morphemes from an analytic and a classifier language, and thereby deepen our understanding of neurobiology of human language.     

Focusing Narrowly or Broadly Attention When Judging Categorical and Coordinate Spatial Relations: A MEG Study

We measured activity in the dorsal system of the human cortex with magnetoencephalography (MEG) during a matching-to-sample plus cueing paradigm, where participants judged the occurrence of changes in either categorical or coordinate spatial relations (e.g., exchanges of left versus right positions or changes in the relative distances) between images of pairs of animals. The attention window was primed in each trial to be either small or large by using cues that immediately preceded the matching image. In this manner, we could assess the modulatory effects of the scope of attention on the activity of the dorsal system of the human cortex during spatial relations processing. The MEG measurements revealed that large spatial cues yielded greater activations and longer peak latencies in the right inferior parietal lobe for coordinate trials, whereas small cues yielded greater activations and longer peak latencies in the left inferior parietal lobe for categorical trials. The activity in the superior parietal lobe, middle frontal gyrus, and visual cortex, was also modulated by the size of the spatial cues and by the type of spatial relation change. The present results support the theory that the lateralization of each kind of spatial processing hinges on differences in the sizes of regions of space attended to by the two hemispheres. In addition, the present findings are inconsistent with the idea of a right-hemispheric dominance for all kinds of challenging spatial tasks, since response times and accuracy rates showed that the categorical spatial relation task was more difficult than the coordinate task and the cortical activations were overall greater in the left hemisphere than in the right hemisphere.     

Rubber hands feel touch, but not in blind individuals

Psychology and neuroscience have a long-standing tradition of studying blind individuals to investigate how visual experience shapes perception of the external world. Here, we study how blind people experience their own body by exposing them to a multisensory body illusion: the somatic rubber hand illusion. In this illusion, healthy blindfolded participants experience that they are touching their own right hand with their left index finger, when in fact they are touching a rubber hand with their left index finger while the experimenter touches their right hand in a synchronized manner (Ehrsson et al. 2005). We compared the strength of this illusion in a group of blind individuals (n = 10), all of whom had experienced severe visual impairment or complete blindness from birth, and a group of age-matched blindfolded sighted participants (n = 12). The illusion was quantified subjectively using questionnaires and behaviorally by asking participants to point to the felt location of the right hand. The results showed that the sighted participants experienced a strong illusion, whereas the blind participants experienced no illusion at all, a difference that was evident in both tests employed. A further experiment testing the participants' basic ability to localize the right hand in space without vision (proprioception) revealed no difference between the two groups. Taken together, these results suggest that blind individuals with impaired visual development have a more veridical percept of self-touch and a less flexible and dynamic representation of their own body in space compared to sighted individuals. We speculate that the multisensory brain systems that re-map somatosensory signals onto external reference frames are less developed in blind individuals and therefore do not allow efficient fusion of tactile and proprioceptive signals from the two upper limbs into a single illusory experience of self-touch as in sighted individuals.     

Shifting attention within memory representations involves early visual areas

Prior studies have shown that spatial attention modulates early visual cortex retinotopically, resulting in enhanced processing of external perceptual representations. However, it is not clear whether the same visual areas are modulated when attention is focused on, and shifted within a working memory representation. In the current fMRI study participants were asked to memorize an array containing four stimuli. After a delay, participants were presented with a verbal cue instructing them to actively maintain the location of one of the stimuli in working memory. Additionally, on a number of trials a second verbal cue instructed participants to switch attention to the location of another stimulus within the memorized representation. Results of the study showed that changes in the BOLD pattern closely followed the locus of attention within the working memory representation. A decrease in BOLD-activity (V1-V3) was observed at ROIs coding a memory location when participants switched away from this location, whereas an increase was observed when participants switched towards this location. Continuous increased activity was obtained at the memorized location when participants did not switch. This study shows that shifting attention within memory representations activates the earliest parts of visual cortex (including V1) in a retinotopic fashion. We conclude that even in the absence of visual stimulation, early visual areas support shifting of attention within memorized representations, similar to when attention is shifted in the outside world. The relationship between visual working memory and visual mental imagery is discussed in light of the current findings.     

Sensitive period for a multimodal response in human visual motion area MT/MST

The middle temporal complex (MT/MST) is a brain region specialized for the perception of motion in the visual modality. However, this specialization is modified by visual experience: after long-standing blindness, MT/MST responds to sound. Recent evidence also suggests that the auditory response of MT/MST is selective for motion. The developmental time course of this plasticity is not known. To test for a sensitive period in MT/MST development, we used fMRI to compare MT/MST function in congenitally blind, late-blind, and sighted adults. MT/MST responded to sound in congenitally blind adults, but not in late-blind or sighted adults, and not in an individual who lost his vision between ages of 2 and 3 years. All blind adults had reduced functional connectivity between MT/MST and other visual regions. Functional connectivity was increased between MT/MST and lateral prefrontal areas in congenitally blind relative to sighted and late-blind adults. These data suggest that early blindness affects the function of feedback projections from prefrontal cortex to MT/MST. We conclude that there is a sensitive period for visual specialization in MT/MST. During typical development, early visual experience either maintains or creates a vision-dominated response. Once established, this response profile is not altered by long-standing blindness.