An fMRI Study of Audiovisual Speech Perception Reveals Multisensory Interactions in Auditory Cortex

Research on the neural basis of speech-reading implicates a network of auditory language regions involving inferior frontal cortex, premotor cortex and sites along superior temporal cortex. In audiovisual speech studies, neural activity is consistently reported in posterior superior temporal Sulcus (pSTS) and this site has been implicated in multimodal integration. Traditionally, multisensory interactions are considered high-level processing that engages heteromodal association cortices (such as STS). Recent work, however, challenges this notion and suggests that multisensory interactions may occur in low-level unimodal sensory cortices. While previous audiovisual speech studies demonstrate that high-level multisensory interactions occur in pSTS, what remains unclear is how early in the processing hierarchy these multisensory interactions may occur. The goal of the present fMRI experiment is to investigate how visual speech can influence activity in auditory cortex above and beyond its response to auditory speech. In an audiovisual speech experiment, subjects were presented with auditory speech with and without congruent visual input. Holding the auditory stimulus constant across the experiment, we investigated how the addition of visual speech influences activity in auditory cortex. We demonstrate that congruent visual speech increases the activity in auditory cortex.     

Using Hypnotic Suggestion to Model Loss of Control and Awareness of Movements: An Exploratory fMRI Study

The feeling of voluntary control and awareness of movement is fundamental to our notions of selfhood and responsibility for actions, yet can be lost in neuropsychiatric syndromes (e.g. delusions of control, non-epileptic seizures) and culturally influenced dissociative states (e.g. attributions of spirit possession). The brain processes involved remain poorly understood. We used suggestion and functional magnetic resonance imaging (fMRI) to investigate loss of control and awareness of right hand movements in 15 highly hypnotically suggestible subjects. Loss of perceived control of movements was associated with reduced connectivity between supplementary motor area (SMA) and motor regions. Reduced awareness of involuntary movements was associated with less activation in parietal cortices (BA 7, BA 40) and insula. Collectively these results suggest that the sense of voluntary control of movement may critically depend on the functional coupling of SMA with motor systems, and provide a potential neural basis for the narrowing of awareness reported in pathological and culturally influenced dissociative phenomena.     

Asymmetries in Perception of 3D Orientation

Visual scene interpretation depends on assumptions based on the statistical regularities of the world. People have some preference for seeing ambiguously oriented objects (Necker cubes) as if tilted down or viewed from above. This bias is a near certainty in the first instant (∼1 s) of viewing and declines over the course of many seconds. In addition, we found that there is modulation of perceived orientation that varies with position—for example objects on the left are more likely to be interpreted as viewed from the right. Therefore there is both a viewed-from-above prior and a scene position-dependent modulation of perceived 3-D orientation. These results are consistent with the idea that ambiguously oriented objects are initially assigned an orientation consistent with our experience of an asymmetric world in which objects most probably sit on surfaces below eye level.     

Real-Time fMRI Pattern Decoding and Neurofeedback Using FRIEND: An FSL-Integrated BCI Toolbox

The demonstration that humans can learn to modulate their own brain activity based on feedback of neurophysiological signals opened up exciting opportunities for fundamental and applied neuroscience. Although EEG-based neurofeedback has been long employed both in experimental and clinical investigation, functional MRI (fMRI)-based neurofeedback emerged as a promising method, given its superior spatial resolution and ability to gauge deep cortical and subcortical brain regions. In combination with improved computational approaches, such as pattern recognition analysis (e.g., Support Vector Machines, SVM), fMRI neurofeedback and brain decoding represent key innovations in the field of neuromodulation and functional plasticity. Expansion in this field and its applications critically depend on the existence of freely available, integrated and user-friendly tools for the neuroimaging research community. Here, we introduce FRIEND, a graphic-oriented user-friendly interface package for fMRI neurofeedback and real-time multivoxel pattern decoding. The package integrates routines for image preprocessing in real-time, ROI-based feedback (single-ROI BOLD level and functional connectivity) and brain decoding-based feedback using SVM. FRIEND delivers an intuitive graphic interface with flexible processing pipelines involving optimized procedures embedding widely validated packages, such as FSL and libSVM. In addition, a user-defined visual neurofeedback module allows users to easily design and run fMRI neurofeedback experiments using ROI-based or multivariate classification approaches. FRIEND is open-source and free for non-commercial use. Processing tutorials and extensive documentation are available.     

Emotion-Motion Interactions in Conversion Disorder: An fMRI Study

Objectives

To evaluate the neural correlates of implicit processing of negative emotions in motor conversion disorder (CD) patients.

Methods

An event related fMRI task was completed by 12 motor CD patients and 14 matched healthy controls using standardised stimuli of faces with fearful and sad emotional expressions in comparison to faces with neutral expressions. Temporal changes in the sensitivity to stimuli were also modelled and tested in the two groups.

Results

We found increased amygdala activation to negative emotions in CD compared to healthy controls in region of interest analyses, which persisted over time consistent with previous findings using emotional paradigms. Furthermore during whole brain analyses we found significantly increased activation in CD patients in areas involved in the ‘freeze response’ to fear (periaqueductal grey matter), and areas involved in self-awareness and motor control (cingulate gyrus and supplementary motor area).

Conclusions

In contrast to healthy controls, CD patients exhibited increased response amplitude to fearful stimuli over time, suggesting abnormal emotional regulation (failure of habituation / sensitization). Patients with CD also activated midbrain and frontal structures that could reflect an abnormal behavioral-motor response to negative including threatening stimuli. This suggests a mechanism linking emotions to motor dysfunction in CD.     

Spatiotemporal Segregation of Neural Response to Auditory Stimulation: An fMRI Study Using Independent Component Analysis and Frequency-Domain Analysis

Although auditory processing has been widely studied with conventional parametric methods, there have been a limited number of independent component analysis (ICA) applications in this area. The purpose of this study was to examine spatiotemporal behavior of brain networks in response to passive auditory stimulation using ICA. Continuous broadband noise was presented binaurally to 19 subjects with normal hearing. ICA was performed to segregate spatial networks, which were subsequently classified according to their temporal relation to the stimulus using power spectrum analysis. Classification of separated networks resulted in 3 stimulus-activated, 9 stimulus-deactivated, 2 stimulus-neutral (stimulus-dependent but not correlated with the stimulation timing), and 2 stimulus-unrelated (fluctuations that did not follow the stimulus cycles) components. As a result of such classification, spatiotemporal subdivisions were observed in a number of cortical structures, namely auditory, cingulate, and sensorimotor cortices, where parts of the same cortical network responded to the stimulus with different temporal patterns. The majority of the classified networks seemed to comprise subparts of the known resting-state networks (RSNs); however, they displayed different temporal behavior in response to the auditory stimulus, indicating stimulus-dependent temporal segregation of RSNs. Only one of nine deactivated networks coincided with the “classic” default-mode network, suggesting the existence of a stimulus-dependent default-mode network, different from that commonly accepted.     

The Neural Correlates of Identity Faking and Concealment: An fMRI Study

The neural basis of self and identity has received extensive research. However, most of these existing studies have focused on situations where the internal representation of the self is consistent with the external one. The present study used fMRI methodology to examine the neural correlates of two different types of identity conflict: identity faking and concealment. Participants were presented with a sequence of names and asked to either conceal their own identity or fake another one. The results revealed that the right insular cortex and bilaterally inferior frontal gyrus were more active for identity concealment compared to the control condition, whereas identity faking elicited a significantly larger percentage signal increase than the control condition in the right superior frontal gyrus, left calcarine, and right caudate. These results suggest that different neural systems associated with both identity processing and deception were involved in identity concealment and faking.     

Lying about the Valence of Affective Pictures: An fMRI Study

The neural correlates of lying about affective information were studied using a functional magnetic resonance imaging (fMRI) methodology. Specifically, 13 healthy right-handed Chinese men were instructed to lie about the valence, positive or negative, of pictures selected from the International Affective Picture System (IAPS) while their brain activity was scanned by a 3T Philip Achieva scanner. The key finding is that the neural activity associated with deception is valence-related. Comparing to telling the truth, deception about the valence of the affectively positive pictures was associated with activity in the inferior frontal, cingulate, inferior parietal, precuneus, and middle temporal regions. Lying about the valence of the affectively negative pictures, on the other hand, was associated with activity in the orbital and medial frontal regions. While a clear valence-related effect on deception was observed, common neural regions were also recruited for the process of deception about the valence of the affective pictures. These regions included the lateral prefrontal and inferior parietal regions. Activity in these regions has been widely reported in fMRI studies on deception using affectively-neutral stimuli. The findings of this study reveal the effect of valence on the neural activity associated with deception. Furthermore, the data also help to illustrate the complexity of the neural mechanisms underlying deception.     

Song Perception by Professional Singers and Actors: An MEG Study

The cortical correlates of speech and music perception are essentially overlapping, and the specific effects of different types of training on these networks remain unknown. We compared two groups of vocally trained professionals for music and speech, singers and actors, using recited and sung rhyme sequences from German art songs with semantic and/ or prosodic/melodic violations (i.e. violations of pitch) of the last word, in order to measure the evoked activation in a magnetoencephalographic (MEG) experiment. MEG data confirmed the existence of intertwined networks for the sung and spoken modality in an early time window after word violation. In essence for this early response, higher activity was measured after melodic/prosodic than semantic violations in predominantly right temporal areas. For singers as well as for actors, modality-specific effects were evident in predominantly left-temporal lateralized activity after semantic expectancy violations in the spoken modality, and right-dominant temporal activity in response to melodic violations in the sung modality. As an indication of a special group-dependent audiation process, higher neuronal activity for singers appeared in a late time window in right temporal and left parietal areas, both after the recited and the sung sequences.     

Coping with Self-Threat and the Evaluation of Self-Related Traits: An fMRI Study

A positive view of oneself is important for a healthy lifestyle. Self-protection mechanisms such as suppressing negative self-related information help us to maintain a positive view of ourselves. This is of special relevance when, for instance, a negative test result threatens our positive self-view. To date, it is not clear which brain areas support self-protective mechanisms under self-threat. In the present functional magnetic resonance imaging (fMRI) study the participants (N = 46) received a (negative vs. positive) performance test feedback before entering the scanner. In the scanner, the participants were instructed to ascribe personality traits either to themselves or to a famous other. Our results showed that participants responded slower to negative self-related traits compared to positive self-related traits. High self-esteem individuals responded slower to negative traits compared to low self-esteem individuals following a self-threat. This indicates that high self-esteem individuals engage more in self-enhancing strategies after a threat by inhibiting negative self-related information more successfully than low self-esteem individuals. This behavioral pattern was mirrored in the fMRI data as dACC correlated positively with trait self-esteem. Generally, ACC activation was attenuated under threat when participants evaluated self-relevant traits and even more for negative self-related traits. We also found that activation in the ACC was negatively correlated with response times, indicating that greater activation of the ACC is linked to better access (faster response) to positive self-related traits and to impaired access (slower response) to negative self-related traits. These results confirm the ACC function as important in managing threatened self-worth but indicate differences in trait self-esteem levels. The fMRI analyses also revealed a decrease in activation within the left Hippocampus and the right thalamus under threat. This indicates that a down-regulation of activation in these regions might also serve as coping mechanism in dealing with self-threat.     

Directed Forgetting of Negative Self-Referential Information Is Difficult: An fMRI Study

A large body of evidence suggested that both emotion and self-referential processing can enhance memory. However, it remains unclear how these two factors influence directed forgetting. This study speculates that directed forgetting of negative self-referential memory is more difficult than forgetting of other-referential memory. To verify this speculation, we combined the directed forgetting paradigm with the self-reference task. The behavioral result suggested that although both self-referential and other-referential information can be directly forgotten, less self-referential information can be forgotten than other-referential information. At the neural level, the forget instruction strongly activated the frontal cortex, suggesting that directed forgetting is not memory decay but an active process. In addition, compared with the negative other-referential information, forgetting of the negative self-referential information were associated with a more widespread activation, including the orbital frontal gyrus (BA47), the inferior frontal gyrus (BA45, BA44), and the middle frontal gyrus. Our results suggest that forgetting of the self-referential information seems to be a more demanding and difficult process.     

A Supramodal Neural Network for Speech and Gesture Semantics: An fMRI Study

In a natural setting, speech is often accompanied by gestures. As language, speech-accompanying iconic gestures to some extent convey semantic information. However, if comprehension of the information contained in both the auditory and visual modality depends on same or different brain-networks is quite unknown. In this fMRI study, we aimed at identifying the cortical areas engaged in supramodal processing of semantic information. BOLD changes were recorded in 18 healthy right-handed male subjects watching video clips showing an actor who either performed speech (S, acoustic) or gestures (G, visual) in more (+) or less (−) meaningful varieties. In the experimental conditions familiar speech or isolated iconic gestures were presented; during the visual control condition the volunteers watched meaningless gestures (G−), while during the acoustic control condition a foreign language was presented (S−). The conjunction of the visual and acoustic semantic processing revealed activations extending from the left inferior frontal gyrus to the precentral gyrus, and included bilateral posterior temporal regions. We conclude that proclaiming this frontotemporal network the brain''s core language system is to take too narrow a view. Our results rather indicate that these regions constitute a supramodal semantic processing network.     

Temporal and Motor Representation of Rhythm in Fronto-Parietal Cortical Areas: An fMRI Study

When sounds occur with temporally structured patterns, we can feel a rhythm. To memorize a rhythm, perception of its temporal patterns and organization of them into a hierarchically structured sequence are necessary. On the other hand, rhythm perception can often cause unintentional body movements. Thus, we hypothesized that rhythm information can be manifested in two different ways; temporal and motor representations. The motor representation depends on effectors, such as the finger or foot, whereas the temporal representation is effector-independent. We tested our hypothesis with a working memory paradigm to elucidate neuronal correlates of temporal or motor representation of rhythm and to reveal the neural networks associated with these representations. We measured brain activity by fMRI while participants memorized rhythms and reproduced them by tapping with the right finger, left finger, or foot, or by articulation. The right inferior frontal gyrus and the inferior parietal lobule exhibited significant effector-independent activations during encoding and retrieval of rhythm information, whereas the left inferior parietal lobule and supplementary motor area (SMA) showed effector-dependent activations during retrieval. These results suggest that temporal sequences of rhythm are probably represented in the right fronto-parietal network, whereas motor sequences of rhythm can be represented in the SMA-parietal network.     

Emotional Picture and Word Processing: An fMRI Study on Effects of Stimulus Complexity

Neuroscientific investigations regarding aspects of emotional experiences usually focus on one stimulus modality (e.g., pictorial or verbal). Similarities and differences in the processing between the different modalities have rarely been studied directly. The comparison of verbal and pictorial emotional stimuli often reveals a processing advantage of emotional pictures in terms of larger or more pronounced emotion effects evoked by pictorial stimuli. In this study, we examined whether this picture advantage refers to general processing differences or whether it might partly be attributed to differences in visual complexity between pictures and words. We first developed a new stimulus database comprising valence and arousal ratings for more than 200 concrete objects representable in different modalities including different levels of complexity: words, phrases, pictograms, and photographs. Using fMRI we then studied the neural correlates of the processing of these emotional stimuli in a valence judgment task, in which the stimulus material was controlled for differences in emotional arousal. No superiority for the pictorial stimuli was found in terms of emotional information processing with differences between modalities being revealed mainly in perceptual processing regions. While visual complexity might partly account for previously found differences in emotional stimulus processing, the main existing processing differences are probably due to enhanced processing in modality specific perceptual regions. We would suggest that both pictures and words elicit emotional responses with no general superiority for either stimulus modality, while emotional responses to pictures are modulated by perceptual stimulus features, such as picture complexity.     

Argument Structure and Morphological Factors in Noun and Verb Processing: An fMRI Study

In a functional MRI (fMRI) study, we have investigated the grammatical categories of object noun, event noun and verb in order to assess the cortical regions of activation supporting their processing. Twelve Italian healthy participants performed a lexical decision task. They had to decide whether a string was an Italian word or not. Words could be objects like medaglia (medal), or events like the noun pianto (cry); or the verb dormire (to sleep). Noun and verb comparison shows differences in regions of activation in the left Inferior Frontal cortex and in the extent of the same areas. We have found specific areas of activation for object noun, and similarities in the pattern of activation for event noun and verb. The activations induced by pseudowords highly resembled the areas activated by the corresponding word category. The implications of the results are discussed in light of the recent debate on the role of grammatical category in the brain.     

Compensatory Effort Parallels Midbrain Deactivation during Mental Fatigue: An fMRI Study

Fatigue reflects the functioning of our physiological negative feedback system, which prevents us from overworking. When fatigued, however, we often try to suppress this system in an effort to compensate for the resulting deterioration in performance. Previous studies have suggested that the effect of fatigue on neurovascular demand may be influenced by this compensatory effort. The primary goal of the present study was to isolate the effect of compensatory effort on neurovascular demand. Healthy male volunteers participated in a series of visual and auditory divided attention tasks that steadily increased fatigue levels for 2 hours. Functional magnetic resonance imaging scans were performed during the first and last quarter of the study (Pre and Post sessions, respectively). Tasks with low and high attentional load (Low and High conditions, respectively) were administrated in alternating blocks. We assumed that compensatory effort would be greater under the High-attentional-load condition compared with the Low-load condition. The difference was assessed during the two sessions. The effect of compensatory effort on neurovascular demand was evaluated by examining the interaction between load (High vs. Low) and time (Pre vs. Post). Significant fatigue-induced deactivation (i.e., Pre>Post) was observed in the frontal, temporal, occipital, and parietal cortices, in the cerebellum, and in the midbrain in both the High and Low conditions. The interaction was significantly greater in the High than in the Low condition in the midbrain. Neither significant fatigue-induced activation (i.e., Pre<Post), nor its interaction with factor Load, was identified. The observed midbrain deactivation ([PreH – PostH]>[PreE– PostE]) may reflect suppression of the negative feedback system that normally triggers recuperative rest to maintain homeostasis.     

Brain Deactivation in the Outperformance in Bimodal Tasks: An fMRI Study

While it is known that some individuals can effectively perform two tasks simultaneously, other individuals cannot. How the brain deals with performing simultaneous tasks remains unclear. In the present study, we aimed to assess which brain areas corresponded to various phenomena in task performance. Nineteen subjects were requested to sequentially perform three blocks of tasks, including two unimodal tasks and one bimodal task. The unimodal tasks measured either visual feature binding or auditory pitch comparison, while the bimodal task required performance of the two tasks simultaneously. The functional magnetic resonance imaging (fMRI) results are compatible with previous studies showing that distinct brain areas, such as the visual cortices, frontal eye field (FEF), lateral parietal lobe (BA7), and medial and inferior frontal lobe, are involved in processing of visual unimodal tasks. In addition, the temporal lobes and Brodmann area 43 (BA43) were involved in processing of auditory unimodal tasks. These results lend support to concepts of modality-specific attention. Compared to the unimodal tasks, bimodal tasks required activation of additional brain areas. Furthermore, while deactivated brain areas were related to good performance in the bimodal task, these areas were not deactivated where the subject performed well in only one of the two simultaneous tasks. These results indicate that efficient information processing does not require some brain areas to be overly active; rather, the specific brain areas need to be relatively deactivated to remain alert and perform well on two tasks simultaneously. Meanwhile, it can also offer a neural basis for biofeedback in training courses, such as courses in how to perform multiple tasks simultaneously.     

Determining the Precise Cerebral Response to Acupuncture: An Improved fMRI Study

Background

In acupuncture brain imaging trials, there are many non-acupuncture factors confounding the neuronal mapping. The modality of the placebo, subjects’ psychological attitude to acupuncture and their physical state are the three most confounding factors.

Objective

To obtain more precise and accurate cerebral fMRI mapping of acupuncture.

Design and Setting

A 2×2 randomized, controlled, participant-blinded cross-over factorial acupuncture trial was conducted at Xuanwu Hospital in Beijing, China.

Participants

Forty-one college students with myopia were recruited to participate in our study and were allocated randomly to four groups, Group A, Group B, Group C and Group D.

Interventions

Group A received real acupuncture (RA) and treatment instruction (TI); Group B received RA and non-treatment instruction (NI); Group C received sham acupuncture (SA) and TI; Group D received SA and NI.

Results

Stimulation at LR3 activated some areas of the visual cortex, and the cerebral response to non-acupuncture factors was complex and occurred in multiple areas.

Conclusions

The results provide more evidence regarding the credibility of acupuncture therapy and suggest that more precise experimental designs are needed to eliminate sources of bias in acupuncture controlled trials and to obtain sound results.