The Perception of Dynamic and Static Facial Expressions of Happiness and Disgust Investigated by ERPs and fMRI Constrained Source Analysis

A recent functional magnetic resonance imaging (fMRI) study by our group demonstrated that dynamic emotional faces are more accurately recognized and evoked more widespread patterns of hemodynamic brain responses than static emotional faces. Based on this experimental design, the present study aimed at investigating the spatio-temporal processing of static and dynamic emotional facial expressions in 19 healthy women by means of multi-channel electroencephalography (EEG), event-related potentials (ERP) and fMRI-constrained regional source analyses. ERP analysis showed an increased amplitude of the LPP (late posterior positivity) over centro-parietal regions for static facial expressions of disgust compared to neutral faces. In addition, the LPP was more widespread and temporally prolonged for dynamic compared to static faces of disgust and happiness. fMRI constrained source analysis on static emotional face stimuli indicated the spatio-temporal modulation of predominantly posterior regional brain activation related to the visual processing stream for both emotional valences when compared to the neutral condition in the fusiform gyrus. The spatio-temporal processing of dynamic stimuli yielded enhanced source activity for emotional compared to neutral conditions in temporal (e.g., fusiform gyrus), and frontal regions (e.g., ventromedial prefrontal cortex, medial and inferior frontal cortex) in early and again in later time windows. The present data support the view that dynamic facial displays trigger more information reflected in complex neural networks, in particular because of their changing features potentially triggering sustained activation related to a continuing evaluation of those faces. A combined fMRI and EEG approach thus provides an advanced insight to the spatio-temporal characteristics of emotional face processing, by also revealing additional neural generators, not identifiable by the only use of an fMRI approach.     

Neural Correlates of Emotional Interference in Social Anxiety Disorder

Disorder-relevant but task-unrelated stimuli impair cognitive performance in social anxiety disorder (SAD); however, time course and neural correlates of emotional interference are unknown. The present study investigated time course and neural basis of emotional interference in SAD using event-related functional magnetic resonance imaging (fMRI). Patients with SAD and healthy controls performed an emotional stroop task which allowed examining interference effects on the current and the succeeding trial. Reaction time data showed an emotional interference effect in the current trial, but not the succeeding trial, specifically in SAD. FMRI data showed greater activation in the left amygdala, bilateral insula, medial prefrontal cortex (mPFC), dorsal anterior cingulate cortex (ACC), and left opercular part of the inferior frontal gyrus during emotional interference of the current trial in SAD patients. Furthermore, we found a positive correlation between patients’ interference scores and activation in the mPFC, dorsal ACC and left angular/supramarginal gyrus. Taken together, results indicate a network of brain regions comprising amygdala, insula, mPFC, ACC, and areas strongly involved in language processing during the processing of task-unrelated threat in SAD. However, specifically the activation in mPFC, dorsal ACC, and left angular/supramarginal gyrus is associated with the strength of the interference effect, suggesting a cognitive network model of attentional bias in SAD. This probably comprises exceeded allocation of attentional resources to disorder-related information of the presented stimuli and increased self-referential and semantic processing of threat words in SAD.     

Abnormal attentional modulation of retinotopic cortex in parietal patients with spatial neglect

Brain regions beyond visual cortex are thought to be responsible for attention-related modulation of visual processing [1, 2], but most evidence is indirect. Here, we applied functional magnetic resonance imaging (fMRI), including retinotopic mapping of visual areas, to patients with focal right-parietal lesions and left spatial neglect [3, 4]. When attentional load at fixation was minimal, retinotopic areas in right visual cortex showed preserved responses to task-irrelevant checkerboards in the contralateral left hemifield, analogously to left visual cortex for right-hemifield checkerboards, indicating a "symmetric" pattern in both hemispheres with respect to contralateral stimulation under these conditions. But when attentional load at fixation was increased, a functional asymmetry emerged for visual cortex, with contralateral responses in right visual areas being pathologically reduced (even eliminated for right V4/TEO), whereas left visual areas showed no such reduction in their contralateral response. These results reveal attention-dependent abnormalities in visual cortex after lesions in distant (parietal) regions. This may explain otherwise puzzling aspects of neglect [5, 6], as confirmed here by additional behavioral testing.     

Early specialization for voice and emotion processing in the infant brain

Human voices play a fundamental role in social communication, and areas of the adult "social brain" show specialization for processing voices and their emotional content (superior temporal sulcus, inferior prefrontal cortex, premotor cortical regions, amygdala, and insula). However, it is unclear when this specialization develops. Functional magnetic resonance (fMRI) studies suggest that the infant temporal cortex does not differentiate speech from music or backward speech, but a prior study with functional near-infrared spectroscopy revealed preferential activation for human voices in 7-month-olds, in a more posterior location of the temporal cortex than in adults. However, the brain networks involved in processing nonspeech human vocalizations in early development are still unknown. To address this issue, in the present fMRI study, 3- to 7-month-olds were presented with adult nonspeech vocalizations (emotionally neutral, emotionally positive, and emotionally negative) and nonvocal environmental sounds. Infants displayed significant differential activation in the anterior portion of the temporal cortex, similarly to adults. Moreover, sad vocalizations modulated the activity of brain regions involved in processing affective stimuli such as the orbitofrontal cortex and insula. These results suggest remarkably early functional specialization for processing human voice and negative emotions.     

Personnalisation des émotions et cortex médial préfrontal

Functional brain imaging studies in healthy subjects suggest that several regions (prefrontal cortex, amygdala, thalamus, hippocampus, anterior cingulate) have specialized functions for emotional operations. Within these regions, the medial prefrontal cortex (MPFC) is considered to have a general role in emotional processing. Using a memory paradigm with verbal material, we recently demonstrated that the MPFC is specifically involved in self-related processing of emotional stimuli. Study with mood induction also suggest that personality traits may modulate the reactivity of the MPFC to emotional Stressors. Taken together these findings support the hypothesis that the MPFC subserve processes involved in emotion regulation. Dysfunction of the MPFC and related structures (i.e. amygdala) may increase the vulnerability to emotional disorders.     

Concurrent TMS-fMRI and psychophysics reveal frontal influences on human retinotopic visual cortex

BACKGROUND: Regions in human frontal cortex may have modulatory top-down influences on retinotopic visual cortex, but to date neuroimaging methods have only been able to provide indirect evidence for such functional interactions between remote but interconnected brain regions. Here we combined transcranial magnetic stimulation (TMS) with concurrent functional magnetic resonance imaging (fMRI), plus psychophysics, to show that stimulation of the right human frontal eye-field (FEF) produced a characteristic topographic pattern of activity changes in retinotopic visual areas V1-V4, with functional consequences for visual perception. RESULTS: FEF TMS led to activity increases for retinotopic representations of the peripheral visual field, but to activity decreases for the central field, in areas V1-V4. These frontal influences on visual cortex occurred in a top-down manner, independently of visual input. TMS of a control site (vertex) did not elicit such visual modulations, and saccades, blinks, or pupil dilation could not account for our results. Finally, the effects of FEF TMS on activity in retinotopic visual cortex led to a behavioral prediction that we confirmed psychophysically by showing that TMS of the frontal site (again compared with vertex) enhanced perceived contrast for peripheral relative to central visual stimuli. CONCLUSIONS: Our results provide causal evidence that circuits originating in the human FEF can modulate activity in retinotopic visual cortex, in a manner that differentiates the central and peripheral visual field, with functional consequences for perception. More generally, our study illustrates how the new approach of concurrent TMS-fMRI can now reveal causal interactions between remote but interconnected areas of the human brain.     

Auditory processing across the sleep-wake cycle: simultaneous EEG and fMRI monitoring in humans

We combined fMRI and EEG recording to study the neurophysiological responses associated with auditory stimulation across the sleep-wake cycle. We found that presentation of auditory stimuli produces bilateral activation in auditory cortex, thalamus, and caudate during both wakefulness and nonrapid eye movement (NREM) sleep. However, the left parietal and, bilaterally, the prefrontal and cingulate cortices and the thalamus were less activated during NREM sleep compared to wakefulness. These areas may play a role in the further processing of sensory information required to achieve conscious perception during wakefulness. Finally, during NREM sleep, the left amygdala and the left prefrontal cortex were more activated by stimuli having special affective significance than by neutral stimuli. These data suggests that the sleeping brain can process auditory stimuli and detect meaningful events.     

Asymmetrical activation in the human brain during processing of fearful faces

Traditional split-field studies and patient research indicate a privileged role for the right hemisphere in emotional processing [1-7], but there has been little direct fMRI evidence for this, despite many studies on emotional-face processing [8-10](see Supplemental Background). With fMRI, we addressed differential hemispheric processing of fearful versus neutral faces by presenting subjects with faces bilaterally [11-13]and orthogonally manipulating whether each hemifield showed a fearful or neutral expression prior to presentation of a checkerboard target. Target discrimination in the left visual field was more accurate after a fearful face was presented there. Event-related fMRI showed right-lateralized brain activations for fearful minus neutral left-hemifield faces in right visual areas, as well as more activity in the right than in the left amygdala. These activations occurred regardless of the type of right-hemifield face shown concurrently, concordant with the behavioral effect. No analogous behavioral or fMRI effects were observed for fearful faces in the right visual field (left hemisphere). The amygdala showed enhanced functional coupling with right-middle and anterior-fusiform areas in the context of a left-hemifield fearful face. These data provide behavioral and fMRI evidence for right-lateralized emotional processing during bilateral stimulation involving enhanced coupling of the amygdala and right-hemispheric extrastriate cortex.     

Attentional modulation strength in cortical area MT depends on stimulus contrast

The attentional modulation of sensory information processing in the visual system is the result of top-down influences, which can cause a multiplicative modulation of the firing rate of sensory neurons in extrastriate visual cortex, an effect reminiscent of the bottom-up effect of changes in stimulus contrast. This similarity could simply reflect the multiplicity of both effects. But, here we show that in direction-selective neurons in monkey visual cortical area MT, stimulus and attentional effects share a nonlinearity. These neurons show higher response gain for both contrast and attentional changes for intermediate contrast stimuli and smaller gain for low- and high-contrast stimuli. This finding suggests a close relationship between the neural encoding of stimulus contrast and the modulating effect of the behavioral relevance of stimuli.     

Brain Metabolism during Hallucination-Like Auditory Stimulation in Schizophrenia

Auditory verbal hallucinations (AVH) in schizophrenia are typically characterized by rich emotional content. Despite the prominent role of emotion in regulating normal perception, the neural interface between emotion-processing regions such as the amygdala and auditory regions involved in perception remains relatively unexplored in AVH. Here, we studied brain metabolism using FDG-PET in 9 remitted patients with schizophrenia that previously reported severe AVH during an acute psychotic episode and 8 matched healthy controls. Participants were scanned twice: (1) at rest and (2) during the perception of aversive auditory stimuli mimicking the content of AVH. Compared to controls, remitted patients showed an exaggerated response to the AVH-like stimuli in limbic and paralimbic regions, including the left amygdala. Furthermore, patients displayed abnormally strong connections between the amygdala and auditory regions of the cortex and thalamus, along with abnormally weak connections between the amygdala and medial prefrontal cortex. These results suggest that abnormal modulation of the auditory cortex by limbic-thalamic structures might be involved in the pathophysiology of AVH and may potentially account for the emotional features that characterize hallucinatory percepts in schizophrenia.     

Neural correlates of individual performance differences in resolving perceptual conflict

Attentional mechanisms are a crucial prerequisite to organize behavior. Most situations may be characterized by a 'competition' between salient, but irrelevant stimuli and less salient, relevant stimuli. In such situations top-down and bottom-up mechanisms interact with each other. In the present fMRI study, we examined how interindividual differences in resolving situations of perceptual conflict are reflected in brain networks mediating attentional selection. Doing so, we employed a change detection task in which subjects had to detect luminance changes in the presence and absence of competing distractors. The results show that good performers presented increased activation in the orbitofrontal cortex (BA 11), anterior cingulate (BA 25), inferior parietal lobule (BA 40) and visual areas V2 and V3 but decreased activation in BA 39. This suggests that areas mediating top-down attentional control are stronger activated in this group. Increased activity in visual areas reflects distinct neuronal enhancement relating to selective attentional mechanisms in order to solve the perceptual conflict. Opposed to good performers, brain areas activated by poor performers comprised the left inferior parietal lobule (BA 39) and fronto-parietal and visual regions were continuously deactivated, suggesting that poor performers perceive stronger conflict than good performers. Moreover, the suppression of neural activation in visual areas might indicate a strategy of poor performers to inhibit the processing of the irrelevant non-target feature. These results indicate that high sensitivity in perceptual areas and increased attentional control led to less conflict in stimulus processing and consequently to higher performance in competitive attentional selection.     

Altered effective connectivity network of the amygdala in social anxiety disorder: a resting-state FMRI study

The amygdala is often found to be abnormally recruited in social anxiety disorder (SAD) patients. The question whether amygdala activation is primarily abnormal and affects other brain systems or whether it responds "normally" to an abnormal pattern of information conveyed by other brain structures remained unanswered. To address this question, we investigated a network of effective connectivity associated with the amygdala using Granger causality analysis on resting-state functional MRI data of 22 SAD patients and 21 healthy controls (HC). Implications of abnormal effective connectivity and clinical severity were investigated using the Liebowitz Social Anxiety Scale (LSAS). Decreased influence from inferior temporal gyrus (ITG) to amygdala was found in SAD, while bidirectional influences between amygdala and visual cortices were increased compared to HCs. Clinical relevance of decreased effective connectivity from ITG to amygdala was suggested by a negative correlation of LSAS avoidance scores and the value of Granger causality. Our study is the first to reveal a network of abnormal effective connectivity of core structures in SAD. This is in support of a disregulation in predescribed modules involved in affect control. The amygdala is placed in a central position of dysfunction characterized both by decreased regulatory influence of orbitofrontal cortex and increased crosstalk with visual cortex. The model which is proposed based on our results lends neurobiological support towards cognitive models considering disinhibition and an attentional bias towards negative stimuli as a core feature of the disorder.     

Increased Amygdala and Visual Cortex Activity and Functional Connectivity towards Stimulus Novelty Is Associated with State Anxiety

Novel stimuli often require a rapid reallocation of sensory processing resources to determine the significance of the event, and the appropriate behavioral response. Both the amygdala and the visual cortex are central elements of the neural circuitry responding to novelty, demonstrating increased activity to new as compared to highly familiarized stimuli. Further, these brain areas are intimately connected, and thus the amygdala may be a key region for directing sensory processing resources to novel events. Although knowledge regarding the neurocircuit of novelty detection is gradually increasing, we still lack a basic understanding of the conditions that are necessary and sufficient for novelty-specific responses in human amygdala and the visual cortices, and if these brain areas interact during detection of novelty. In the present study, we investigated the response of amygdala and the visual cortex to novelty, by comparing functional MRI activity between 1^st and 2^nd time presentation of a series of emotional faces in an event-related task. We observed a significant decrease in amygdala and visual cortex activity already after a single stimulus exposure. Interestingly, this decrease in responsiveness was less for subjects with a high score on state anxiety. Further, novel faces stimuli were associated with a relative increase in the functional coupling between the amygdala and the inferior occipital gyrus (BA 18). Thus, we suggest that amygdala is involved in fast sensory boosting that may be important for attention reallocation to novel events, and that the strength of this response depends on individual state anxiety.     

Eyes Wide Shut: Amygdala Mediates Eyes-Closed Effect on Emotional Experience with Music

The perceived emotional value of stimuli and, as a consequence the subjective emotional experience with them, can be affected by context-dependent styles of processing. Therefore, the investigation of the neural correlates of emotional experience requires accounting for such a variable, a matter of an experimental challenge. Closing the eyes affects the style of attending to auditory stimuli by modifying the perceptual relationship with the environment without changing the stimulus itself. In the current study, we used fMRI to characterize the neural mediators of such modification on the experience of emotionality in music. We assumed that closed eyes position will reveal interplay between different levels of neural processing of emotions. More specifically, we focused on the amygdala as a central node of the limbic system and on its co-activation with the Locus Ceruleus (LC) and Ventral Prefrontal Cortex (VPFC); regions involved in processing of, respectively, ‘low’, visceral-, and ‘high’, cognitive-related, values of emotional stimuli. Fifteen healthy subjects listened to negative and neutral music excerpts with eyes closed or open. As expected, behavioral results showed that closing the eyes while listening to emotional music resulted in enhanced rating of emotionality, specifically of negative music. In correspondence, fMRI results showed greater activation in the amygdala when subjects listened to the emotional music with eyes closed relative to eyes open. More so, by using voxel-based correlation and a dynamic causal model analyses we demonstrated that increased amygdala activation to negative music with eyes closed led to increased activations in the LC and VPFC. This finding supports a system-based model of perceived emotionality in which the amygdala has a central role in mediating the effect of context-based processing style by recruiting neural operations involved in both visceral (i.e. ‘low’) and cognitive (i.e. ‘high’) related processes of emotions.     

The Emotional Gatekeeper: A Computational Model of Attentional Selection and Suppression through the Pathway from the Amygdala to the Inhibitory Thalamic Reticular Nucleus

In a complex environment that contains both opportunities and threats, it is important for an organism to flexibly direct attention based on current events and prior plans. The amygdala, the hub of the brain''s emotional system, is involved in forming and signaling affective associations between stimuli and their consequences. The inhibitory thalamic reticular nucleus (TRN) is a hub of the attentional system that gates thalamo-cortical signaling. In the primate brain, a recently discovered pathway from the amygdala sends robust projections to TRN. Here we used computational modeling to demonstrate how the amygdala-TRN pathway, embedded in a wider neural circuit, can mediate selective attention guided by emotions. Our Emotional Gatekeeper model demonstrates how this circuit enables focused top-down, and flexible bottom-up, allocation of attention. The model suggests that the amygdala-TRN projection can serve as a unique mechanism for emotion-guided selection of signals sent to cortex for further processing. This inhibitory selection mechanism can mediate a powerful affective ‘framing’ effect that may lead to biased decision-making in highly charged emotional situations. The model also supports the idea that the amygdala can serve as a relevance detection system. Further, the model demonstrates how abnormal top-down drive and dysregulated local inhibition in the amygdala and in the cortex can contribute to the attentional symptoms that accompany several neuropsychiatric disorders.     

The neuropsychology of face perception: beyond simple dissociations and functional selectivity

Face processing relies on a distributed, patchy network of cortical regions in the temporal and frontal lobes that respond disproportionately to face stimuli, other cortical regions that are not even primarily visual (such as somatosensory cortex), and subcortical structures such as the amygdala. Higher-level face perception abilities, such as judging identity, emotion and trustworthiness, appear to rely on an intact face-processing network that includes the occipital face area (OFA), whereas lower-level face categorization abilities, such as discriminating faces from objects, can be achieved without OFA, perhaps via the direct connections to the fusiform face area (FFA) from several extrastriate cortical areas. Some lesion, transcranial magnetic stimulation (TMS) and functional magnetic resonance imaging (fMRI) findings argue against a strict feed-forward hierarchical model of face perception, in which the OFA is the principal and common source of input for other visual and non-visual cortical regions involved in face perception, including the FFA, face-selective superior temporal sulcus and somatosensory cortex. Instead, these findings point to a more interactive model in which higher-level face perception abilities depend on the interplay between several functionally and anatomically distinct neural regions. Furthermore, the nature of these interactions may depend on the particular demands of the task. We review the lesion and TMS literature on this topic and highlight the dynamic and distributed nature of face processing.     

Ability to Maintain Internal Arousal and Motivation Modulates Brain Responses to Emotions

Persistence (PS) is defined as the ability to generate and maintain arousal and motivation internally in the absence of immediate external reward. Low PS individuals tend to become discouraged when expectations are not rapidly fulfilled. The goal of this study was to investigate whether individual differences in PS influence the recruitment of brain regions involved in emotional processing and regulation. In a functional MRI study, 35 subjects judged the emotional intensity of displayed pictures. When processing negative pictures, low PS (vs. high PS) subjects showed higher amygdala and right orbito-frontal cortex (OFC) activity but lower left OFC activity. This dissociation in OFC activity suggests greater prefrontal cortical asymmetry for approach/avoidance motivation, suggesting an avoidance response to aversive stimuli in low PS. For positive or neutral stimuli, low PS subjects showed lower activity in the amygdala, striatum, and hippocampus. These results suggest that low PS may involve an imbalance in processing distinct emotional inputs, with greater reactivity to aversive information in regions involved in avoidance behaviour (amygdala, OFC) and dampened response to positive and neutral stimuli across circuits subserving motivated behaviour (striatum, hippocampus, amygdala). Low PS affective style was associated with depression vulnerability. These findings in non-depressed subjects point to a neural mechanism whereby some individuals are more likely to show systematic negative emotional biases, as frequently observed in depression. The assessment of these individual differences, including those that may cause vulnerability to depressive disorders, would therefore constitute a promising approach to risk assessment for depression.     

Effects of attention and emotion on face processing in the human brain: an event-related fMRI study.

We used event-related fMRI to assess whether brain responses to fearful versus neutral faces are modulated by spatial attention. Subjects performed a demanding matching task for pairs of stimuli at prespecified locations, in the presence of task-irrelevant stimuli at other locations. Faces or houses unpredictably appeared at the relevant or irrelevant locations, while the faces had either fearful or neutral expressions. Activation of fusiform gyri by faces was strongly affected by attentional condition, but the left amygdala response to fearful faces was not. Right fusiform activity was greater for fearful than neutral faces, independently of the attention effect on this region. These results reveal differential influences on face processing from attention and emotion, with the amygdala response to threat-related expressions unaffected by a manipulation of attention that strongly modulates the fusiform response to faces.     

Selective attention to task-irrelevant emotional distractors is unaffected by the perceptual load associated with a foreground task

A number of studies have shown that emotionally arousing stimuli are preferentially processed in the human brain. Whether or not this preference persists under increased perceptual load associated with a task at hand remains an open question. Here we manipulated two possible determinants of the attentional selection process, perceptual load associated with a foreground task and the emotional valence of concurrently presented task-irrelevant distractors. As a direct measure of sustained attentional resource allocation in early visual cortex we used steady-state visual evoked potentials (SSVEPs) elicited by distinct flicker frequencies of task and distractor stimuli. Subjects either performed a detection (low load) or discrimination (high load) task at a centrally presented symbol stream that flickered at 8.6 Hz while task-irrelevant neutral or unpleasant pictures from the International Affective Picture System (IAPS) flickered at a frequency of 12 Hz in the background of the stream. As reflected in target detection rates and SSVEP amplitudes to both task and distractor stimuli, unpleasant relative to neutral background pictures more strongly withdrew processing resources from the foreground task. Importantly, this finding was unaffected by the factor 'load' which turned out to be a weak modulator of attentional processing in human visual cortex.     

Hemispheric specialization for global and local processing: the effect of stimulus category.

Neuropsychological evidence indicates that the global aspect of complex visual scenes is preferentially processed by the right hemisphere, and local aspects are preferentially processed by the left hemisphere. Using letter-based hierarchical stimuli (Navon figures), we recently demonstrated, in a directed-attention task, lateralized neural activity (assessed by positron emission tomography) in the left prestriate cortex during local processing, and in the right prestriate cortex during global processing. Furthermore, temporal-parietal cortex was critically activated bilaterally in a divided-attention task that involved varying the number of target switches between local and global levels of letter-based hierarchical stimuli. Little is known about whether such stimulus categories influence such hemispheric lateralization. We now present data on brain activity, derived from positron emission tomography, in normal subjects scanned during either local or global processing of object-based hierarchical stimuli. We again observe attentional modulation of neural activity in prestriate cortex. There is now greater right-sided activation for local processing and greater left-sided activation for global processing, which is the opposite of that seen with letter-based stimuli. The results suggest that the relative differential hemispheric activations in the prestriate areas during global and local processing are modified by stimulus category.