Evaluation of a shape-based model of human face discrimination using FMRI and behavioral techniques

Understanding the neural mechanisms underlying object recognition is one of the fundamental challenges of visual neuroscience. While neurophysiology experiments have provided evidence for a "simple-to-complex" processing model based on a hierarchy of increasingly complex image features, behavioral and fMRI studies of face processing have been interpreted as incompatible with this account. We present a neurophysiologically plausible, feature-based model that quantitatively accounts for face discrimination characteristics, including face inversion and "configural" effects. The model predicts that face discrimination is based on a sparse representation of units selective for face shapes, without the need to postulate additional, "face-specific" mechanisms. We derive and test predictions that quantitatively link model FFA face neuron tuning, neural adaptation measured in an fMRI rapid adaptation paradigm, and face discrimination performance. The experimental data are in excellent agreement with the model prediction that discrimination performance should asymptote as faces become dissimilar enough to activate different neuronal populations.     

Asymmetrical activation in the human brain during processing of fearful faces

Traditional split-field studies and patient research indicate a privileged role for the right hemisphere in emotional processing [1-7], but there has been little direct fMRI evidence for this, despite many studies on emotional-face processing [8-10](see Supplemental Background). With fMRI, we addressed differential hemispheric processing of fearful versus neutral faces by presenting subjects with faces bilaterally [11-13]and orthogonally manipulating whether each hemifield showed a fearful or neutral expression prior to presentation of a checkerboard target. Target discrimination in the left visual field was more accurate after a fearful face was presented there. Event-related fMRI showed right-lateralized brain activations for fearful minus neutral left-hemifield faces in right visual areas, as well as more activity in the right than in the left amygdala. These activations occurred regardless of the type of right-hemifield face shown concurrently, concordant with the behavioral effect. No analogous behavioral or fMRI effects were observed for fearful faces in the right visual field (left hemisphere). The amygdala showed enhanced functional coupling with right-middle and anterior-fusiform areas in the context of a left-hemifield fearful face. These data provide behavioral and fMRI evidence for right-lateralized emotional processing during bilateral stimulation involving enhanced coupling of the amygdala and right-hemispheric extrastriate cortex.     

Face inversion reduces the persistence of global form and its neural correlates

Face inversion produces a detrimental effect on face recognition. The extent to which the inversion of faces and other kinds of objects influences the perceptual binding of visual information into global forms is not known. We used a behavioral method and functional MRI (fMRI) to measure the effect of face inversion on visual persistence, a type of perceptual memory that reflects sustained awareness of global form. We found that upright faces persisted longer than inverted versions of the same images; we observed a similar effect of inversion on the persistence of animal stimuli. This effect of inversion on persistence was evident in sustained fMRI activity throughout the ventral visual hierarchy, including the lateral occipital area (LO), two face-selective visual areas--the fusiform face area (FFA) and the occipital face area (OFA)--and several early visual areas. V1 showed the same initial fMRI activation to upright and inverted forms but this activation lasted longer for upright stimuli. The inversion effect on persistence-related fMRI activity in V1 and other retinotopic visual areas demonstrates that higher-tier visual areas influence early visual processing via feedback. This feedback effect on figure-ground processing is sensitive to the orientation of the figure.     

The neuropsychology of face perception: beyond simple dissociations and functional selectivity

Face processing relies on a distributed, patchy network of cortical regions in the temporal and frontal lobes that respond disproportionately to face stimuli, other cortical regions that are not even primarily visual (such as somatosensory cortex), and subcortical structures such as the amygdala. Higher-level face perception abilities, such as judging identity, emotion and trustworthiness, appear to rely on an intact face-processing network that includes the occipital face area (OFA), whereas lower-level face categorization abilities, such as discriminating faces from objects, can be achieved without OFA, perhaps via the direct connections to the fusiform face area (FFA) from several extrastriate cortical areas. Some lesion, transcranial magnetic stimulation (TMS) and functional magnetic resonance imaging (fMRI) findings argue against a strict feed-forward hierarchical model of face perception, in which the OFA is the principal and common source of input for other visual and non-visual cortical regions involved in face perception, including the FFA, face-selective superior temporal sulcus and somatosensory cortex. Instead, these findings point to a more interactive model in which higher-level face perception abilities depend on the interplay between several functionally and anatomically distinct neural regions. Furthermore, the nature of these interactions may depend on the particular demands of the task. We review the lesion and TMS literature on this topic and highlight the dynamic and distributed nature of face processing.     

The Hierarchical Brain Network for Face Recognition

Numerous functional magnetic resonance imaging (fMRI) studies have identified multiple cortical regions that are involved in face processing in the human brain. However, few studies have characterized the face-processing network as a functioning whole. In this study, we used fMRI to identify face-selective regions in the entire brain and then explore the hierarchical structure of the face-processing network by analyzing functional connectivity among these regions. We identified twenty-five regions mainly in the occipital, temporal and frontal cortex that showed a reliable response selective to faces (versus objects) across participants and across scan sessions. Furthermore, these regions were clustered into three relatively independent sub-networks in a face-recognition task on the basis of the strength of functional connectivity among them. The functionality of the sub-networks likely corresponds to the recognition of individual identity, retrieval of semantic knowledge and representation of emotional information. Interestingly, when the task was switched to object recognition from face recognition, the functional connectivity between the inferior occipital gyrus and the rest of the face-selective regions were significantly reduced, suggesting that this region may serve as an entry node in the face-processing network. In sum, our study provides empirical evidence for cognitive and neural models of face recognition and helps elucidate the neural mechanisms underlying face recognition at the network level.     

Face perception: domain specific, not process specific

Evidence that face perception is mediated by special cognitive and neural mechanisms comes from fMRI studies of the fusiform face area (FFA) and behavioral studies of the face inversion effect. Here, we used these two methods to ask whether face perception mechanisms are stimulus specific, process specific, or both. Subjects discriminated pairs of upright or inverted faces or house stimuli that differed in either the spatial distance among parts (configuration) or the shape of the parts. The FFA showed a much higher response to faces than to houses, but no preference for the configuration task over the part task. Similarly, the behavioral inversion effect was as large in the part task as the configuration task for faces, but absent in both part and configuration tasks for houses. These findings indicate that face perception mechanisms are not process specific for parts or configuration but are domain specific for face stimuli per se.     

Human face processing is tuned to sexual age preferences

Human faces can motivate nurturing behaviour or sexual behaviour when adults see a child or an adult face, respectively. This suggests that face processing is tuned to detecting age cues of sexual maturity to stimulate the appropriate reproductive behaviour: either caretaking or mating. In paedophilia, sexual attraction is directed to sexually immature children. Therefore, we hypothesized that brain networks that normally are tuned to mature faces of the preferred gender show an abnormal tuning to sexual immature faces in paedophilia. Here, we use functional magnetic resonance imaging (fMRI) to test directly for the existence of a network which is tuned to face cues of sexual maturity. During fMRI, participants sexually attracted to either adults or children were exposed to various face images. In individuals attracted to adults, adult faces activated several brain regions significantly more than child faces. These brain regions comprised areas known to be implicated in face processing, and sexual processing, including occipital areas, the ventrolateral prefrontal cortex and, subcortically, the putamen and nucleus caudatus. The same regions were activated in paedophiles, but with a reversed preferential response pattern.     

Non-invasive evaluation of face and motion perception in humans

The neural mechanisms for the perception of face and motion were studied using psychophysical threshold measurements, event-related potentials (ERPs), and functional magnetic resonance imaging (fMRI). A face-specific ERP component, N170, was recorded over the posterior temporal cortex. Removal of the high-spatial-frequency components of the face altered the perception of familiar faces significantly, and familiarity can facilitate the cortico-cortical processing of facial perceptions. Similarly, the high-spatial-frequency components of the face seemed to be crucial for the recognition of facial expressions. Aging and visuospatial impairments affected motion perception significantly. Two distinct components of motion ERPs, N170 and P200, were recorded over the parietal region. The former was related to horizontal motion perception while the latter reflected the perception of radial optic flow motion. The results of fMRI showed that horizontal movements of objects and radial optic flow motion were perceived differently in the V5/MT and superior parietal lobe. We conclude that an integrated approach can provide useful information on spatial and temporal processing of face and motion non-invasively.     

The effect of face inversion on activity in human neural systems for face and object perception

The differential effect of stimulus inversion on face and object recognition suggests that inverted faces are processed by mechanisms for the perception of other objects rather than by face perception mechanisms. We investigated the face inversion using functional magnetic resonance imaging (fMRI). The principal effect of face inversion on was an increased response in ventral extrastriate regions that respond preferentially to another class of objects (houses). In contrast, house inversion did not produce a similar change in face-selective regions. Moreover, stimulus inversion had equivalent, minimal effects for faces in in face-selective regions and for houses in house-selective regions. The results suggest that the failure of face perception systems with inverted faces leads to the recruitment of processing resources in object perception systems, but this failure is not reflected by altered activity in face perception systems.     

The Perception of Dynamic and Static Facial Expressions of Happiness and Disgust Investigated by ERPs and fMRI Constrained Source Analysis

A recent functional magnetic resonance imaging (fMRI) study by our group demonstrated that dynamic emotional faces are more accurately recognized and evoked more widespread patterns of hemodynamic brain responses than static emotional faces. Based on this experimental design, the present study aimed at investigating the spatio-temporal processing of static and dynamic emotional facial expressions in 19 healthy women by means of multi-channel electroencephalography (EEG), event-related potentials (ERP) and fMRI-constrained regional source analyses. ERP analysis showed an increased amplitude of the LPP (late posterior positivity) over centro-parietal regions for static facial expressions of disgust compared to neutral faces. In addition, the LPP was more widespread and temporally prolonged for dynamic compared to static faces of disgust and happiness. fMRI constrained source analysis on static emotional face stimuli indicated the spatio-temporal modulation of predominantly posterior regional brain activation related to the visual processing stream for both emotional valences when compared to the neutral condition in the fusiform gyrus. The spatio-temporal processing of dynamic stimuli yielded enhanced source activity for emotional compared to neutral conditions in temporal (e.g., fusiform gyrus), and frontal regions (e.g., ventromedial prefrontal cortex, medial and inferior frontal cortex) in early and again in later time windows. The present data support the view that dynamic facial displays trigger more information reflected in complex neural networks, in particular because of their changing features potentially triggering sustained activation related to a continuing evaluation of those faces. A combined fMRI and EEG approach thus provides an advanced insight to the spatio-temporal characteristics of emotional face processing, by also revealing additional neural generators, not identifiable by the only use of an fMRI approach.     

The neural basis of the behavioral face-inversion effect

Two of the most robust markers for "special" face processing are the behavioral face-inversion effect (FIE)-the disproportionate drop in recognition of upside-down (inverted) stimuli relative to upright faces-and the face-selective fMRI response in the fusiform face area (FFA). However, the relationship between these two face-selective markers is unknown. Here we report that the behavioral FIE is closely associated with the fMRI response in the FFA, but not in other face-selective or object-selective regions. The FFA and the face-selective region in the superior temporal sulcus (f_STS), but not the occipital face-selective region (OFA), showed a higher response to upright than inverted faces. However, only in the FFA was this fMRI-FIE positively correlated across subjects with the behavioral FIE. Second, the FFA, but not the f_STS, showed greater neural sensitivity to differences between faces when they were upright than inverted, suggesting a possible neural mechanism for the behavioral FIE. Although a similar trend was found in the occipital face area (OFA), it was less robust than the FFA. Taken together, our data suggest that among the face-selective and object-selective regions, the FFA is a primary neural source of the behavioral FIE.     

Influence of Repressive Coping Style on Cortical Activation during Encoding of Angry Faces

Background

Coping plays an important role for emotion regulation in threatening situations. The model of coping modes designates repression and sensitization as two independent coping styles. Repression consists of strategies that shield the individual from arousal. Sensitization indicates increased analysis of the environment in order to reduce uncertainty. According to the discontinuity hypothesis, repressors are sensitive to threat in the early stages of information processing. While repressors do not exhibit memory disturbances early on, they manifest weak memory for these stimuli later. This study investigates the discontinuity hypothesis using functional magnetic resonance imaging (fMRI).

Methods

Healthy volunteers (20 repressors and 20 sensitizers) were selected from a sample of 150 students on the basis of the Mainz Coping Inventory. During the fMRI experiment, subjects evaluated and memorized emotional and neutral faces. Subjects performed two sessions of face recognition: immediately after the fMRI session and three days later.

Results

Repressors exhibited greater activation of frontal, parietal and temporal areas during encoding of angry faces compared to sensitizers. There were no differences in recognition of facial emotions between groups neither immediately after exposure nor after three days.

Conclusions

The fMRI findings suggest that repressors manifest an enhanced neural processing of directly threatening facial expression which confirms the assumption of hyper-responsivity to threatening information in repression in an early processing stage. A discrepancy was observed between high neural activation in encoding-relevant brain areas in response to angry faces in repressors and no advantage in subsequent memory for these faces compared to sensitizers.     

Teaching about Scientific Origins: Taking Account of Creationism

The purpose of this study was to develop a model for measuring experimental design ability based on functional magnetic resonance imaging (fMRI) during biological inquiry. More specifically, the researchers developed an experimental design task that measures experimental design ability. Using the developed experimental design task, they measured both the paper experimental design ability and the fMRI experimental design ability of subjects. Subjects’ paper experimental design ability was measured using the quotient equation of experimental design ability, and their fMRI experimental design ability using the brain connectivity coefficient. According to the fMRI results, differences in design ability existed among subjects in terms of brain connectivity coefficient level during the experimental design task. The experimental design ability brain connectivity coefficient level and quotient for each subject were analysed. Statistically significant correlations between subjects’ connectivity strength level among brain activation regions and quotient value guided the establishment of a measuring model. The model measured experimental design ability and could predict an individual’s experimental design ability quotient using his or her brain connectivity coefficient. Hence, the model developed for this study for measuring experimental design ability based on fMRI may serve as a practical measurement of students’ scientific experimental design ability. Furthermore, this study could serve as a founding theory for measuring models of other scientific processing abilities such as observation, question generation, classification, hypothesis generation and hypothesis evaluation.     

Neural correlates of perceiving emotional faces and bodies in developmental prosopagnosia: an event-related fMRI-study

Many people experience transient difficulties in recognizing faces but only a small number of them cannot recognize their family members when meeting them unexpectedly. Such face blindness is associated with serious problems in everyday life. A better understanding of the neuro-functional basis of impaired face recognition may be achieved by a careful comparison with an equally unique object category and by a adding a more realistic setting involving neutral faces as well facial expressions. We used event-related functional magnetic resonance imaging (fMRI) to investigate the neuro-functional basis of perceiving faces and bodies in three developmental prosopagnosics (DP) and matched healthy controls. Our approach involved materials consisting of neutral faces and bodies as well as faces and bodies expressing fear or happiness. The first main result is that the presence of emotional information has a different effect in the patient vs. the control group in the fusiform face area (FFA). Neutral faces trigger lower activation in the DP group, compared to the control group, while activation for facial expressions is the same in both groups. The second main result is that compared to controls, DPs have increased activation for bodies in the inferior occipital gyrus (IOG) and for neutral faces in the extrastriate body area (EBA), indicating that body and face sensitive processes are less categorically segregated in DP. Taken together our study shows the importance of using naturalistic emotional stimuli for a better understanding of developmental face deficits.     

Forging a path to mesoscopic imaging success with ultra-high field functional magnetic resonance imaging

Functional magnetic resonance imaging (fMRI) studies with ultra-high field (UHF, 7+ Tesla) technology enable the acquisition of high-resolution images. In this work, we discuss recent achievements in UHF fMRI at the mesoscopic scale, on the order of cortical columns and layers, and examine approaches to addressing common challenges. As researchers push to smaller and smaller voxel sizes, acquisition and analysis decisions have greater potential to degrade spatial accuracy, and UHF fMRI data must be carefully interpreted. We consider the impact of acquisition decisions on the spatial specificity of the MR signal with a representative dataset with 0.8 mm isotropic resolution. We illustrate the trade-offs in contrast with noise ratio and spatial specificity of different acquisition techniques and show that acquisition blurring can increase the effective voxel size by as much as 50% in some dimensions. We further describe how different sources of degradations to spatial resolution in functional data may be characterized. Finally, we emphasize that progress in UHF fMRI depends not only on scientific discovery and technical advancement, but also on informal discussions and documentation of challenges researchers face and overcome in pursuit of their goals.This article is part of the theme issue ‘Key relationships between non-invasive functional neuroimaging and the underlying neuronal activity’.     

Formal learning theory dissociates brain regions with different temporal integration

Learning can be characterized as the extraction of reliable predictions about stimulus occurrences from past experience. In two experiments, we investigated the interval of temporal integration of previous learning trials in different brain regions using implicit and explicit Pavlovian fear conditioning with a dynamically changing reinforcement regime in an experimental setting. With formal learning theory (the Rescorla-Wagner model), temporal integration is characterized by the learning rate. Using fMRI and this theoretical framework, we are able to distinguish between learning-related brain regions that show long temporal integration (e.g., amygdala) and higher perceptual regions that integrate only over a short period of time (e.g., fusiform face area, parahippocampal place area). This approach allows for the investigation of learning-related changes in brain activation, as it can dissociate brain areas that differ with respect to their integration of past learning experiences by either computing long-term outcome predictions or instantaneous reinforcement expectancies.     

A Review of fMRI Simulation Studies

Simulation studies that validate statistical techniques for fMRI data are challenging due to the complexity of the data. Therefore, it is not surprising that no common data generating process is available (i.e. several models can be found to model BOLD activation and noise). Based on a literature search, a database of simulation studies was compiled. The information in this database was analysed and critically evaluated focusing on the parameters in the simulation design, the adopted model to generate fMRI data, and on how the simulation studies are reported. Our literature analysis demonstrates that many fMRI simulation studies do not report a thorough experimental design and almost consistently ignore crucial knowledge on how fMRI data are acquired. Advice is provided on how the quality of fMRI simulation studies can be improved.     

A general probabilistic model for group independent component analysis and its estimation methods

Independent component analysis (ICA) has become an important tool for analyzing data from functional magnetic resonance imaging (fMRI) studies. ICA has been successfully applied to single-subject fMRI data. The extension of ICA to group inferences in neuroimaging studies, however, is challenging due to the unavailability of a prespecified group design matrix and the uncertainty in between-subjects variability in fMRI data. We present a general probabilistic ICA (PICA) model that can accommodate varying group structures of multisubject spatiotemporal processes. An advantage of the proposed model is that it can flexibly model various types of group structures in different underlying neural source signals and under different experimental conditions in fMRI studies. A maximum likelihood (ML) method is used for estimating this general group ICA model. We propose two expectation-maximization (EM) algorithms to obtain the ML estimates. The first method is an exact EM algorithm, which provides an exact E-step and an explicit noniterative M-step. The second method is a variational approximation EM algorithm, which is computationally more efficient than the exact EM. In simulation studies, we first compare the performance of the proposed general group PICA model and the existing probabilistic group ICA approach. We then compare the two proposed EM algorithms and show the variational approximation EM achieves comparable accuracy to the exact EM with significantly less computation time. An fMRI data example is used to illustrate application of the proposed methods.     

Accounting for movement increases sensitivity in detecting brain activity in Parkinson's disease

Parkinson's disease (PD) is manifested by motor impairment, which may impede the ability to accurately perform motor tasks during functional magnetic resonance imaging (fMRI). Both temporal and amplitude deviations of movement performance affect the blood oxygenation level-dependent (BOLD) response. We present a general approach for assessing PD patients' movement control employing simultaneously recorded fMRI time series and behavioral data of the patients' kinematics using MR-compatible gloves. Twelve male patients with advanced PD were examined with fMRI at 1.5T during epoch-based visually paced finger tapping. MR-compatible gloves were utilized online to quantify motor outcome in two conditions with or without dopaminergic medication. Modeling of individual-level brain activity included (i) a predictor consisting of a condition-specific, constant-amplitude boxcar function convolved with the canonical hemodynamic response function (HRF) as commonly used in fMRI statistics (standard model), or (ii) a custom-made predictor computed from glove time series convolved with the HRF (kinematic model). Factorial statistics yielded a parametric map for each modeling technique, showing the medication effect on the group level. Patients showed bilateral response to levodopa in putamen and globus pallidus during the motor experiment. Interestingly, kinematic modeling produced significantly higher activation in terms of both the extent and amplitude of activity. Our results appear to account for movement performance in fMRI motor experiments with PD and increase sensitivity in detecting brain response to levodopa. We strongly advocate quantitatively controlling for motor performance to reach more reliable and robust analyses in fMRI with PD patients.     

Optimizing Stimulation and Analysis Protocols for Neonatal fMRI

The development of brain function in young infants is poorly understood. The core challenge is that infants have a limited behavioral repertoire through which brain function can be expressed. Neuroimaging with fMRI has great potential as a way of characterizing typical development, and detecting abnormal development early. But, a number of methodological challenges must first be tackled to improve the robustness and sensitivity of neonatal fMRI. A critical one of these, addressed here, is that the hemodynamic response function (HRF) in pre-term and term neonates differs from that in adults, which has a number of implications for fMRI. We created a realistic model of noise in fMRI data, using resting-state fMRI data from infants and adults, and then conducted simulations to assess the effect of HRF of the power of different stimulation protocols and analysis assumptions (HRF modeling). We found that neonatal fMRI is most powerful if block-durations are kept at the lower range of those typically used in adults (full on/off cycle duration 25-30s). Furthermore, we show that it is important to use the age-appropriate HRF during analysis, as mismatches can lead to reduced power or even inverted signal. Where the appropriate HRF is not known (for example due to potential developmental delay), a flexible basis set performs well, and allows accurate post-hoc estimation of the HRF.