Tracking the mind's image in the brain I: time-resolved fMRI during visuospatial mental imagery

Mental imagery, the generation and manipulation of mental representations in the absence of sensory stimulation, is a core element of numerous cognitive processes. We investigate the cortical mechanisms underlying imagery and spatial analysis in the visual domain using event-related functional magnetic resonance imaging during the mental clock task. The time-resolved analysis of cortical activation from auditory perception to motor response reveals a sequential activation of the left and right posterior parietal cortex, suggesting that these regions perform distinct functions in this imagery task. This is confirmed by a trial-by-trial analysis of correlations between reaction time and onset, width, and amplitude of the hemodynamic response. These findings pose neurophysiological constraints on cognitive models of mental imagery.     

Functional neuroanatomy of the hypnotic state

The neural mechanisms underlying hypnosis and especially the modulation of pain perception by hypnosis remain obscure. Using PET we first described the distribution of regional cerebral blood flow during the hypnotic state. Hypnosis relied on revivification of pleasant autobiographical memories and was compared to imaging autobiographical material in "normal alertness". The hypnotic state was related to the activation of a widespread set of cortical areas involving occipital, parietal, precentral, premotor, and ventrolateral prefrontal and anterior cingulate cortices. This pattern of activation shares some similarities with mental imagery, from which it mainly differs by the relative deactivation of precuneus. Second, we looked at the anti-nociceptive effects of hypnosis. Compared to the resting state, hypnosis reduced pain perception by approximately 50%. The hypnosis-induced reduction of affective and sensory responses to noxious thermal stimulation were modulated by the activity in the midcingulate cortex (area 24a'). Finally, we assessed changes in cerebral functional connectivity related to hypnosis. Compared to normal alertness (i.e., rest and mental imagery), the hypnotic state, significantly enhanced the functional modulation between midcingulate cortex and a large neural network involved in sensory, affective, cognitive and behavioral aspects of nociception. These findings show that not only pharmacological but also psychological strategies for pain control can modulate the cerebral network involved in noxious perception.     

Distributed neural systems for the generation of visual images

Visual perception of houses, faces, and chairs evoke differential responses in ventral temporal cortex. Using fMRI, we compared activations evoked by perception and imagery of these object categories. We found content-related activation during imagery in extrastriate cortex, but this activity was restricted to small subsets of the regions that showed category-related activation during perception. Within ventral temporal cortex, activation during imagery evoked stronger responses on the left whereas perception evoked stronger responses on the right. Additionally, visual imagery evoked activity in parietal and frontal cortex, but this activity was not content related. These results suggest that content-related activation during imagery in visual extrastriate cortex may be implemented by "top-down" mechanisms in parietal and frontal cortex that mediate the retrieval of face and object representations from long-term memory and their maintenance through visual imagery.     

Reliability and representational bandwidth in the auditory cortex

It is unclear why there are so many more neurons in sensory cortex than in the sensory periphery. One possibility is that these "extra" neurons are used to overcome cortical noise and faithfully represent the acoustic stimulus. Another possibility is that even after overcoming cortical noise, there is "excess representational bandwidth" available and that this bandwidth is used to represent conjunctions of auditory and nonauditory information for computation. Here, we discuss recent data about neuronal reliability in auditory cortex showing that cortical noise may not be as high as was previously believed. Although at present, the data suggest that auditory cortex neurons can be more reliable than those in the visual cortex, we speculate that the principles governing cortical computation are universal and that visual and other cortical areas can also exploit strategies based on similarly high-fidelity activity.     

Sound-driven synaptic inhibition in primary visual cortex

Multimodal objects and events activate many sensory cortical areas simultaneously. This is possibly reflected in reciprocal modulations of neuronal activity, even at the level of primary cortical areas. However, the synaptic character of these interareal interactions, and their impact on synaptic and behavioral sensory responses are unclear. Here, we found that activation of auditory cortex by a noise burst drove local GABAergic inhibition on supragranular pyramids of the mouse primary visual cortex, via cortico-cortical connections. This inhibition was generated by sound-driven excitation of a limited number of cells in infragranular visual cortical neurons. Consequently, visually driven synaptic and spike responses were reduced upon bimodal stimulation. Also, acoustic stimulation suppressed conditioned behavioral responses to a dim flash, an effect that was prevented by acute blockade of GABAergic transmission in visual cortex. Thus, auditory cortex activation by salient stimuli degrades potentially distracting sensory processing in visual cortex by recruiting local, translaminar, inhibitory circuits.     

Recruitment of occipital cortex during sensory substitution training linked to subjective experience of seeing in people with blindness

Over three months of intensive training with a tactile stimulation device, 18 blind and 10 blindfolded seeing subjects improved in their ability to identify geometric figures by touch. Seven blind subjects spontaneously reported 'visual qualia', the subjective sensation of seeing flashes of light congruent with tactile stimuli. In the latter subjects tactile stimulation evoked activation of occipital cortex on electroencephalography (EEG). None of the blind subjects who failed to experience visual qualia, despite identical tactile stimulation training, showed EEG recruitment of occipital cortex. None of the blindfolded seeing humans reported visual-like sensations during tactile stimulation. These findings support the notion that the conscious experience of seeing is linked to the activation of occipital brain regions in people with blindness. Moreover, the findings indicate that provision of visual information can be achieved through non-visual sensory modalities which may help to minimize the disability of blind individuals, affording them some degree of object recognition and navigation aid.     

Auditory cortex mapmaking: principles, projections, and plasticity

Maps of sensory receptor epithelia and computed features of the sensory environment are common elements of auditory, visual, and somatic sensory representations from the periphery to the cerebral cortex. Maps enhance the understanding of normal neural organization and its modification by pathology and experience. They underlie the derivation of the computational principles that govern sensory processing and the generation of perception. Despite their intuitive explanatory power, the functions of and rules for organizing maps and their plasticity are not well understood. Some puzzles of auditory cortical map organization are that few complete receptor maps are available and that even fewer computational maps are known beyond primary cortical areas. Neuroanatomical evidence suggests equally organized connectional patterns throughout the cortical hierarchy that might underlie map stability. Here, we consider the implications of auditory cortical map organization and its plasticity and evaluate the complementary role of maps in representation and computation from an auditory perspective.     

The effects of neck flexion on cerebral potentials evoked by visual,auditory and somatosensory stimuli and focal brain blood flow in related sensory cortices

Background

A flexed neck posture leads to non-specific activation of the brain. Sensory evoked cerebral potentials and focal brain blood flow have been used to evaluate the activation of the sensory cortex. We investigated the effects of a flexed neck posture on the cerebral potentials evoked by visual, auditory and somatosensory stimuli and focal brain blood flow in the related sensory cortices.

Methods

Twelve healthy young adults received right visual hemi-field, binaural auditory and left median nerve stimuli while sitting with the neck in a resting and flexed (20° flexion) position. Sensory evoked potentials were recorded from the right occipital region, Cz in accordance with the international 10–20 system, and 2 cm posterior from C4, during visual, auditory and somatosensory stimulations. The oxidative-hemoglobin concentration was measured in the respective sensory cortex using near-infrared spectroscopy.

Results

Latencies of the late component of all sensory evoked potentials significantly shortened, and the amplitude of auditory evoked potentials increased when the neck was in a flexed position. Oxidative-hemoglobin concentrations in the left and right visual cortices were higher during visual stimulation in the flexed neck position. The left visual cortex is responsible for receiving the visual information. In addition, oxidative-hemoglobin concentrations in the bilateral auditory cortex during auditory stimulation, and in the right somatosensory cortex during somatosensory stimulation, were higher in the flexed neck position.

Conclusions

Visual, auditory and somatosensory pathways were activated by neck flexion. The sensory cortices were selectively activated, reflecting the modalities in sensory projection to the cerebral cortex and inter-hemispheric connections.     

Contrast and Strength of Visual Memory and Imagery Differentially Affect Visual Perception

Visual short-term memory (VSTM) and visual imagery have been shown to modulate visual perception. However, how the subjective experience of VSTM/imagery and its contrast modulate this process has not been investigated. We addressed this issue by asking participants to detect brief masked targets while they were engaged either in VSTM or visual imagery. Subjective experience of memory/imagery (strength scale), and the visual contrast of the memory/mental image (contrast scale) were assessed on a trial-by-trial basis. For both VSTM and imagery, contrast of the memory/mental image was positively associated with reporting target presence. Consequently, at the sensory level, both VSTM and imagery facilitated visual perception. However, subjective strength of VSTM was positively associated with visual detection whereas the opposite pattern was found for imagery. Thus the relationship between subjective strength of memory/imagery and visual detection are qualitatively different for VSTM and visual imagery, although their impact at the sensory level appears similar. Our results furthermore demonstrate that imagery and VSTM are partly dissociable processes.     

Distortions of subjective time perception within and across senses

Background

The ability to estimate the passage of time is of fundamental importance for perceptual and cognitive processes. One experience of time is the perception of duration, which is not isomorphic to physical duration and can be distorted by a number of factors. Yet, the critical features generating these perceptual shifts in subjective duration are not understood.

Methodology/Findings

We used prospective duration judgments within and across sensory modalities to examine the effect of stimulus predictability and feature change on the perception of duration. First, we found robust distortions of perceived duration in auditory, visual and auditory-visual presentations despite the predictability of the feature changes in the stimuli. For example, a looming disc embedded in a series of steady discs led to time dilation, whereas a steady disc embedded in a series of looming discs led to time compression. Second, we addressed whether visual (auditory) inputs could alter the perception of duration of auditory (visual) inputs. When participants were presented with incongruent audio-visual stimuli, the perceived duration of auditory events could be shortened or lengthened by the presence of conflicting visual information; however, the perceived duration of visual events was seldom distorted by the presence of auditory information and was never perceived shorter than their actual durations.

Conclusions/Significance

These results support the existence of multisensory interactions in the perception of duration and, importantly, suggest that vision can modify auditory temporal perception in a pure timing task. Insofar as distortions in subjective duration can neither be accounted for by the unpredictability of an auditory, visual or auditory-visual event, we propose that it is the intrinsic features of the stimulus that critically affect subjective time distortions.     

Distributed representation of perceptual categories in the auditory cortex

Categorical perception is a process by which a continuous stimulus space is partitioned to represent discrete sensory events. Early experience has been shown to shape categorical perception and enlarge cortical representations of experienced stimuli in the sensory cortex. The present study examines the hypothesis that enlargement in cortical stimulus representations is a mechanism of categorical perception. Perceptual discrimination and identification behaviors were analyzed in model auditory cortices that incorporated sound exposure-induced plasticity effects. The model auditory cortex with over-representations of specific stimuli exhibited categorical perception behaviors for those specific stimuli. These results indicate that enlarged stimulus representations in the sensory cortex may be a mechanism for categorical perceptual learning.     

Visual Cross-Modal Re-Organization in Children with Cochlear Implants

Background

Visual cross-modal re-organization is a neurophysiological process that occurs in deafness. The intact sensory modality of vision recruits cortical areas from the deprived sensory modality of audition. Such compensatory plasticity is documented in deaf adults and animals, and is related to deficits in speech perception performance in cochlear-implanted adults. However, it is unclear whether visual cross-modal re-organization takes place in cochlear-implanted children and whether it may be a source of variability contributing to speech and language outcomes. Thus, the aim of this study was to determine if visual cross-modal re-organization occurs in cochlear-implanted children, and whether it is related to deficits in speech perception performance.

Methods

Visual evoked potentials (VEPs) were recorded via high-density EEG in 41 normal hearing children and 14 cochlear-implanted children, aged 5–15 years, in response to apparent motion and form change. Comparisons of VEP amplitude and latency, as well as source localization results, were conducted between the groups in order to view evidence of visual cross-modal re-organization. Finally, speech perception in background noise performance was correlated to the visual response in the implanted children.

Results

Distinct VEP morphological patterns were observed in both the normal hearing and cochlear-implanted children. However, the cochlear-implanted children demonstrated larger VEP amplitudes and earlier latency, concurrent with activation of right temporal cortex including auditory regions, suggestive of visual cross-modal re-organization. The VEP N1 latency was negatively related to speech perception in background noise for children with cochlear implants.

Conclusion

Our results are among the first to describe cross modal re-organization of auditory cortex by the visual modality in deaf children fitted with cochlear implants. Our findings suggest that, as a group, children with cochlear implants show evidence of visual cross-modal recruitment, which may be a contributing source of variability in speech perception outcomes with their implant.     

Nursing stimulation is more than tactile sensation: It is a multisensory experience

Novel sensory experiences, particularly those associated with epochal developmental events like nursing alter cortical representation, affecting memory, perception and behavior. Functional MRI was used here to test whether the sensoricortical map of the ventrum is modified during lactation. Three stimuli were used to drive cortical activation in primiparous rats: natural, artificial suckling stimulation and general mechanical rubbing of the skin of the ventrum. These stimuli significantly activated the somatosensory cortex of dams. Of the three stimuli, artificial and pup suckling robustly activated much of the cerebrum, most notably the visual, auditory and olfactory cortices. Surprisingly, activation occurred even in the absence of pups, with artificial suckling. This finding suggests that incoming information from a single modality was sufficient to drive activity of others. Enhanced sensitivity across the cortical mantle during nursing may help the dam to perceive, process, and remember stimuli critical to the care and protection of her young.     

The functional impact of mental imagery on conscious perception

Mental imagery has been proposed to contribute to a variety of high-level cognitive functions, including memory encoding and retrieval, navigation, spatial planning, and even social communication and language comprehension. However, it is debated whether mental imagery relies on the same sensory representations as perception, and if so, what functional consequences such an overlap might have on perception itself. We report novel evidence that single instances of imagery can have a pronounced facilitatory influence on subsequent conscious perception. Either seeing or imagining a specific pattern could strongly bias which of two competing stimuli reach awareness during binocular rivalry. Effects of imagery and perception were location and orientation specific, accumulated in strength over time, and survived an intervening visual task lasting several seconds prior to presentation of the rivalry display. Interestingly, effects of imagery differed from those of feature-based attention. The results demonstrate that imagery, in the absence of any incoming visual signals, leads to the formation of a short-term sensory trace that can bias future perception, suggesting a means by which high-level processes that support imagination and memory retrieval may shape low-level sensory representations.     

Auditory cortex basal activity modulates cochlear responses in chinchillas

Background

The auditory efferent system has unique neuroanatomical pathways that connect the cerebral cortex with sensory receptor cells. Pyramidal neurons located in layers V and VI of the primary auditory cortex constitute descending projections to the thalamus, inferior colliculus, and even directly to the superior olivary complex and to the cochlear nucleus. Efferent pathways are connected to the cochlear receptor by the olivocochlear system, which innervates outer hair cells and auditory nerve fibers. The functional role of the cortico-olivocochlear efferent system remains debated. We hypothesized that auditory cortex basal activity modulates cochlear and auditory-nerve afferent responses through the efferent system.

Methodology/Principal Findings

Cochlear microphonics (CM), auditory-nerve compound action potentials (CAP) and auditory cortex evoked potentials (ACEP) were recorded in twenty anesthetized chinchillas, before, during and after auditory cortex deactivation by two methods: lidocaine microinjections or cortical cooling with cryoloops. Auditory cortex deactivation induced a transient reduction in ACEP amplitudes in fifteen animals (deactivation experiments) and a permanent reduction in five chinchillas (lesion experiments). We found significant changes in the amplitude of CM in both types of experiments, being the most common effect a CM decrease found in fifteen animals. Concomitantly to CM amplitude changes, we found CAP increases in seven chinchillas and CAP reductions in thirteen animals. Although ACEP amplitudes were completely recovered after ninety minutes in deactivation experiments, only partial recovery was observed in the magnitudes of cochlear responses.

Conclusions/Significance

These results show that blocking ongoing auditory cortex activity modulates CM and CAP responses, demonstrating that cortico-olivocochlear circuits regulate auditory nerve and cochlear responses through a basal efferent tone. The diversity of the obtained effects suggests that there are at least two functional pathways from the auditory cortex to the cochlea.     

A Synchrony-Dependent Influence of Sounds on Activity in Visual Cortex Measured Using Functional Near-Infrared Spectroscopy (fNIRS)

Evidence from human neuroimaging and animal electrophysiological studies suggests that signals from different sensory modalities interact early in cortical processing, including in primary sensory cortices. The present study aimed to test whether functional near-infrared spectroscopy (fNIRS), an emerging, non-invasive neuroimaging technique, is capable of measuring such multisensory interactions. Specifically, we tested for a modulatory influence of sounds on activity in visual cortex, while varying the temporal synchrony between trains of transient auditory and visual events. Related fMRI studies have consistently reported enhanced activation in response to synchronous compared to asynchronous audiovisual stimulation. Unexpectedly, we found that synchronous sounds significantly reduced the fNIRS response from visual cortex, compared both to asynchronous sounds and to a visual-only baseline. It is possible that this suppressive effect of synchronous sounds reflects the use of an efficacious visual stimulus, chosen for consistency with previous fNIRS studies. Discrepant results may also be explained by differences between studies in how attention was deployed to the auditory and visual modalities. The presence and relative timing of sounds did not significantly affect performance in a simultaneously conducted behavioral task, although the data were suggestive of a positive relationship between the strength of the fNIRS response from visual cortex and the accuracy of visual target detection. Overall, the present findings indicate that fNIRS is capable of measuring multisensory cortical interactions. In multisensory research, fNIRS can offer complementary information to the more established neuroimaging modalities, and may prove advantageous for testing in naturalistic environments and with infant and clinical populations.     

Behavioral and neuroplastic changes in the blind: evidence for functionally relevant cross-modal interactions.

The study of blind individuals provides insight into the brain re-organization and behavioral compensations that occur following sensory deprivation. While behavioral studies have yielded conflicting results in terms of performance levels within the remaining senses, deafferentation of visual cortical areas through peripheral blindness results in clear neuroplastic changes. Most striking is the activation of occipital cortex in response to auditory and tactile stimulation. Indeed, parts of the "unimodal" visual cortex are recruited by other sensory modalities to process sensory information in a functionally relevant manner. In addition, a larger area of the sensorimotor cortex is devoted to the representation of the reading finger in blind Braille readers. The "visual" function of the deafferented occipital cortex is also altered, where transcranial magnetic stimulation-induced phosphenes can be elicited in only 20% of blind subjects. The neural mechanisms underlying these changes remain elusive but recent data showing rapid cross-modal plasticity in blindfolded, sighted subjects argue against the establishment of new connections to explain cross-modal interactions in the blind. Rather, latent pathways that participate in multisensory percepts in sighted subjects might be unmasked and may be potentiated in the event of complete loss of visual input. These issues have important implications for the development of visual prosthesis aimed at restoring some degree of vision in the blind.     

Positron Emission Tomography Imaging Reveals Auditory and Frontal Cortical Regions Involved with Speech Perception and Loudness Adaptation

Considerable progress has been made in the treatment of hearing loss with auditory implants. However, there are still many implanted patients that experience hearing deficiencies, such as limited speech understanding or vanishing perception with continuous stimulation (i.e., abnormal loudness adaptation). The present study aims to identify specific patterns of cerebral cortex activity involved with such deficiencies. We performed O-15-water positron emission tomography (PET) in patients implanted with electrodes within the cochlea, brainstem, or midbrain to investigate the pattern of cortical activation in response to speech or continuous multi-tone stimuli directly inputted into the implant processor that then delivered electrical patterns through those electrodes. Statistical parametric mapping was performed on a single subject basis. Better speech understanding was correlated with a larger extent of bilateral auditory cortex activation. In contrast to speech, the continuous multi-tone stimulus elicited mainly unilateral auditory cortical activity in which greater loudness adaptation corresponded to weaker activation and even deactivation. Interestingly, greater loudness adaptation was correlated with stronger activity within the ventral prefrontal cortex, which could be up-regulated to suppress the irrelevant or aberrant signals into the auditory cortex. The ability to detect these specific cortical patterns and differences across patients and stimuli demonstrates the potential for using PET to diagnose auditory function or dysfunction in implant patients, which in turn could guide the development of appropriate stimulation strategies for improving hearing rehabilitation. Beyond hearing restoration, our study also reveals a potential role of the frontal cortex in suppressing irrelevant or aberrant activity within the auditory cortex, and thus may be relevant for understanding and treating tinnitus.     

Integrating information from different senses in the auditory cortex

Multisensory integration was once thought to be the domain of brain areas high in the cortical hierarchy, with early sensory cortical fields devoted to unisensory processing of inputs from their given set of sensory receptors. More recently, a wealth of evidence documenting visual and somatosensory responses in auditory cortex, even as early as the primary fields, has changed this view of cortical processing. These multisensory inputs may serve to enhance responses to sounds that are accompanied by other sensory cues, effectively making them easier to hear, but may also act more selectively to shape the receptive field properties of auditory cortical neurons to the location or identity of these events. We discuss the new, converging evidence that multiplexing of neural signals may play a key role in informatively encoding and integrating signals in auditory cortex across multiple sensory modalities. We highlight some of the many open research questions that exist about the neural mechanisms that give rise to multisensory integration in auditory cortex, which should be addressed in future experimental and theoretical studies.     

Towards a functional neuroanatomy of conscious perception and its modulation by volition: implications of human auditory neuroimaging studies.

Conscious sensory perception and its modulation by volition are integral to human mental life. Functional neuroimaging techniques provide a direct means of identifying and characterizing in vivo the systems-level patterns of brain activity associated with such mental functions. In a series of positron emission tomography activation experiments, we and our colleagues have examined a range of normal and abnormal auditory states that, when contrasted, provide dissociations relevant to the question of the neural substrates of sensory awareness. These dissociations include sensory awareness in the presence and absence of external sensory stimuli, the transition from sensory unawareness to awareness (or vice versa) in the presence of sensory stimuli, and sensory awareness with and without volition. The auditory states studied include hallucinations, mental imagery, cortical deafness modulated by attention, and hearing modulated by sedation. The results of these studies highlight the distributed nature of the functional neuroanatomy that is sufficient, if not necessary, for sensory awareness. The probable roles of unimodal association (as compared with primary) cortices, heteromodal cortices, limbic/paralimbic regions and subcortical structures (such as the thalamus) are discussed. In addition, interactions between pre- and post-rolandic regions are examined in the context of top-down, volitional modulation of sensory awareness.