The neuropsychology of face perception: beyond simple dissociations and functional selectivity

Face processing relies on a distributed, patchy network of cortical regions in the temporal and frontal lobes that respond disproportionately to face stimuli, other cortical regions that are not even primarily visual (such as somatosensory cortex), and subcortical structures such as the amygdala. Higher-level face perception abilities, such as judging identity, emotion and trustworthiness, appear to rely on an intact face-processing network that includes the occipital face area (OFA), whereas lower-level face categorization abilities, such as discriminating faces from objects, can be achieved without OFA, perhaps via the direct connections to the fusiform face area (FFA) from several extrastriate cortical areas. Some lesion, transcranial magnetic stimulation (TMS) and functional magnetic resonance imaging (fMRI) findings argue against a strict feed-forward hierarchical model of face perception, in which the OFA is the principal and common source of input for other visual and non-visual cortical regions involved in face perception, including the FFA, face-selective superior temporal sulcus and somatosensory cortex. Instead, these findings point to a more interactive model in which higher-level face perception abilities depend on the interplay between several functionally and anatomically distinct neural regions. Furthermore, the nature of these interactions may depend on the particular demands of the task. We review the lesion and TMS literature on this topic and highlight the dynamic and distributed nature of face processing.     

Stimulation of category-selective brain areas modulates ERP to their preferred categories

Neural selectivity to specific object categories has been demonstrated in extrastriate cortex with both functional MRI [1-3] and event-related potential (ERP) [4, 5]. Here we tested for a causal relationship between the activation of category-selective areas and ERP to their preferred categories. Electroencephalogram (EEG) was recorded while participants observed faces and headless bodies. Concurrently with EEG recording, we delivered two pulses of transcranial magnetic stimulation (TMS) over the right occipital face area (OFA) or extrastriate body area (EBA) at 60 and 100 ms after stimulus onset. Results showed a clear dissociation between the stimulated site and the stimulus category on ERP modulation: stimulation of the OFA significantly increased the N1 amplitude to faces but not to bodies, whereas stimulation of the EBA significantly increased the N1 amplitude to bodies but not to faces. These findings provide the first evidence for a specific and causal link between activity in category-selective networks and scalp-recorded ERP to their preferred categories. This result also demonstrates that the face and body N1 reflects several nonoverlapping neural sources, rather than changes in face-selective mechanisms alone. Lastly, because early stimulation (60-100 ms) affected selectivity of a later ERP component (150-200 ms), the results could imply a feed-forward connection between occipital and temporal category-selective areas.     

Feeling by sight or seeing by touch?

We have addressed the role of occipital and somatosensory cortex in a tactile discrimination task. Sight-ed and congenitally blind subjects rated the roughness and distance spacing for a series of raised dot patterns. When judging roughness, intermediate dot spacings were perceived as being the most rough, while distance judgments generated a linear relation. Low-frequency rTMS applied to somatosensory cortex disrupted roughness without affecting distance judgments, while rTMS to occipital cortex disrupted distance but not roughness judgments. We also tested an early blind patient with bilateral occipital cortex damage. Her performance on the roughness determination task was normal; however, she was greatly impaired with distance judgments. The findings suggest a double-dissociation effect in which roughness and distance are primarily processed in somatosensory and occipital cortex, respectively. The differential effect of rTMS on task performance and corroborative clinical evidence suggest that occipital cortex is engaged in tactile tasks requiring fine spatial discrimination.     

The fusiform face area is engaged in holistic, not parts-based, representation of faces

Numerous studies with functional magnetic resonance imaging have shown that the fusiform face area (FFA) in the human brain plays a key role in face perception. Recent studies have found that both the featural information of faces (e.g., eyes, nose, and mouth) and the configural information of faces (i.e., spatial relation among features) are encoded in the FFA. However, little is known about whether the featural information is encoded independent of or combined with the configural information in the FFA. Here we used multi-voxel pattern analysis to examine holistic representation of faces in the FFA by correlating spatial patterns of activation with behavioral performance in discriminating face parts with face configurations either present or absent. Behaviorally, the absence of face configurations (versus presence) impaired discrimination of face parts, suggesting a holistic representation in the brain. Neurally, spatial patterns of activation in the FFA were more similar among correct than incorrect trials only when face parts were presented in a veridical face configuration. In contrast, spatial patterns of activation in the occipital face area, as well as the object-selective lateral occipital complex, were more similar among correct than incorrect trials regardless of the presence of veridical face configurations. This finding suggests that in the FFA faces are represented not on the basis of individual parts but in terms of the whole that emerges from the parts.     

The neural basis of the behavioral face-inversion effect

Two of the most robust markers for "special" face processing are the behavioral face-inversion effect (FIE)-the disproportionate drop in recognition of upside-down (inverted) stimuli relative to upright faces-and the face-selective fMRI response in the fusiform face area (FFA). However, the relationship between these two face-selective markers is unknown. Here we report that the behavioral FIE is closely associated with the fMRI response in the FFA, but not in other face-selective or object-selective regions. The FFA and the face-selective region in the superior temporal sulcus (f_STS), but not the occipital face-selective region (OFA), showed a higher response to upright than inverted faces. However, only in the FFA was this fMRI-FIE positively correlated across subjects with the behavioral FIE. Second, the FFA, but not the f_STS, showed greater neural sensitivity to differences between faces when they were upright than inverted, suggesting a possible neural mechanism for the behavioral FIE. Although a similar trend was found in the occipital face area (OFA), it was less robust than the FFA. Taken together, our data suggest that among the face-selective and object-selective regions, the FFA is a primary neural source of the behavioral FIE.     

Face inversion reduces the persistence of global form and its neural correlates

Face inversion produces a detrimental effect on face recognition. The extent to which the inversion of faces and other kinds of objects influences the perceptual binding of visual information into global forms is not known. We used a behavioral method and functional MRI (fMRI) to measure the effect of face inversion on visual persistence, a type of perceptual memory that reflects sustained awareness of global form. We found that upright faces persisted longer than inverted versions of the same images; we observed a similar effect of inversion on the persistence of animal stimuli. This effect of inversion on persistence was evident in sustained fMRI activity throughout the ventral visual hierarchy, including the lateral occipital area (LO), two face-selective visual areas--the fusiform face area (FFA) and the occipital face area (OFA)--and several early visual areas. V1 showed the same initial fMRI activation to upright and inverted forms but this activation lasted longer for upright stimuli. The inversion effect on persistence-related fMRI activity in V1 and other retinotopic visual areas demonstrates that higher-tier visual areas influence early visual processing via feedback. This feedback effect on figure-ground processing is sensitive to the orientation of the figure.     

The evolution of face processing in primates

The ability to recognize faces is an important socio-cognitive skill that is associated with a number of cognitive specializations in humans. While numerous studies have examined the presence of these specializations in non-human primates, species where face recognition would confer distinct advantages in social situations, results have been mixed. The majority of studies in chimpanzees support homologous face-processing mechanisms with humans, but results from monkey studies appear largely dependent on the type of testing methods used. Studies that employ passive viewing paradigms, like the visual paired comparison task, report evidence of similarities between monkeys and humans, but tasks that use more stringent, operant response tasks, like the matching-to-sample task, often report species differences. Moreover, the data suggest that monkeys may be less sensitive than chimpanzees and humans to the precise spacing of facial features, in addition to the surface-based cues reflected in those features, information that is critical for the representation of individual identity. The aim of this paper is to provide a comprehensive review of the available data from face-processing tasks in non-human primates with the goal of understanding the evolution of this complex cognitive skill.     

Sensory competition in the face processing areas of the human brain

The concurrent presentation of multiple stimuli in the visual field may trigger mutually suppressive interactions throughout the ventral visual stream. While several studies have been performed on sensory competition effects among non-face stimuli relatively little is known about the interactions in the human brain for multiple face stimuli. In the present study we analyzed the neuronal basis of sensory competition in an event-related functional magnetic resonance imaging (fMRI) study using multiple face stimuli. We varied the ratio of faces and phase-noise images within a composite display with a constant number of peripheral stimuli, thereby manipulating the competitive interactions between faces. For contralaterally presented stimuli we observed strong competition effects in the fusiform face area (FFA) bilaterally and in the right lateral occipital area (LOC), but not in the occipital face area (OFA), suggesting their different roles in sensory competition. When we increased the spatial distance among pairs of faces the magnitude of suppressive interactions was reduced in the FFA. Surprisingly, the magnitude of competition depended on the visual hemifield of the stimuli: ipsilateral stimulation reduced the competition effects somewhat in the right LOC while it increased them in the left LOC. This suggests a left hemifield dominance of sensory competition. Our results support the sensory competition theory in the processing of multiple faces and suggests that sensory competition occurs in several cortical areas in both cerebral hemispheres.     

Improved Discrimination of Visual Stimuli Following Repetitive Transcranial Magnetic Stimulation

Background

Repetitive transcranial magnetic stimulation (rTMS) at certain frequencies increases thresholds for motor-evoked potentials and phosphenes following stimulation of cortex. Consequently rTMS is often assumed to introduce a “virtual lesion” in stimulated brain regions, with correspondingly diminished behavioral performance.

Methodology/Principal Findings

Here we investigated the effects of rTMS to visual cortex on subjects'' ability to perform visual psychophysical tasks. Contrary to expectations of a visual deficit, we find that rTMS often improves the discrimination of visual features. For coarse orientation tasks, discrimination of a static stimulus improved consistently following theta-burst stimulation of the occipital lobe. Using a reaction-time task, we found that these improvements occurred throughout the visual field and lasted beyond one hour post-rTMS. Low-frequency (1 Hz) stimulation yielded similar improvements. In contrast, we did not find consistent effects of rTMS on performance in a fine orientation discrimination task.

Conclusions/Significance

Overall our results suggest that rTMS generally improves or has no effect on visual acuity, with the nature of the effect depending on the type of stimulation and the task. We interpret our results in the context of an ideal-observer model of visual perception.     

The Hierarchical Brain Network for Face Recognition

Numerous functional magnetic resonance imaging (fMRI) studies have identified multiple cortical regions that are involved in face processing in the human brain. However, few studies have characterized the face-processing network as a functioning whole. In this study, we used fMRI to identify face-selective regions in the entire brain and then explore the hierarchical structure of the face-processing network by analyzing functional connectivity among these regions. We identified twenty-five regions mainly in the occipital, temporal and frontal cortex that showed a reliable response selective to faces (versus objects) across participants and across scan sessions. Furthermore, these regions were clustered into three relatively independent sub-networks in a face-recognition task on the basis of the strength of functional connectivity among them. The functionality of the sub-networks likely corresponds to the recognition of individual identity, retrieval of semantic knowledge and representation of emotional information. Interestingly, when the task was switched to object recognition from face recognition, the functional connectivity between the inferior occipital gyrus and the rest of the face-selective regions were significantly reduced, suggesting that this region may serve as an entry node in the face-processing network. In sum, our study provides empirical evidence for cognitive and neural models of face recognition and helps elucidate the neural mechanisms underlying face recognition at the network level.     

Dissociation of category versus item priming in face processing: an event-related potential study

The underlying specificity of visual object categorization and discrimination can be elucidated by studying different types of repetition priming. Here we focused on this issue in face processing. We investigated category priming (i.e. the prime and target stimuli represent different exemplars of the same object category) and item priming (i.e. the prime and target stimuli are exactly the same image), using an immediate repetition paradigm. Twenty-three subjects were asked to respond as fast and accurately as possible to categorize whether the target stimulus was a face or a building image, but to ignore the prime stimulus. We recorded event-related potentials (ERPs) and reaction times (RTs) simultaneously. The RT data showed significant effects of category priming in both face trials and building trials, as well as a significant effect of item priming in face trials. With respect to the ERPs, in face trials, no priming effect was observed at the P100 stage, whereas a category priming effect emerged at the N170 stage, and an item priming effect at the P200 stage. In contrast, in building trials, priming effects occurred already at the P100 stage. Our results indicated that distinct neural mechanisms underlie separable kinds of immediate repetition priming in face processing.     

Explicating the Face Perception Network with White Matter Connectivity

A network of multiple brain regions is recruited in face perception. Our understanding of the functional properties of this network can be facilitated by explicating the structural white matter connections that exist between its functional nodes. We accomplished this using functional MRI (fMRI) in combination with fiber tractography on high angular resolution diffusion weighted imaging data. We identified the three nodes of the core face network: the “occipital face area” (OFA), the “fusiform face area” (mid-fusiform gyrus or mFus), and the superior temporal sulcus (STS). Additionally, a region of the anterior temporal lobe (aIT), implicated as being important for face perception was identified. Our data suggest that we can further divide the OFA into multiple anatomically distinct clusters – a partitioning consistent with several recent neuroimaging results. More generally, structural white matter connectivity within this network revealed: 1) Connectivity between aIT and mFus, and between aIT and occipital regions, consistent with studies implicating this posterior to anterior pathway as critical to normal face processing; 2) Strong connectivity between mFus and each of the occipital face-selective regions, suggesting that these three areas may subserve different functional roles; 3) Almost no connectivity between STS and mFus, or between STS and the other face-selective regions. Overall, our findings suggest a re-evaluation of the “core” face network with respect to what functional areas are or are not included in this network.     

Echolocation sound features processed to provide distance information in the CF/FM bat,Noctilio albiventris: evidence for a gated time window utilizing both CF and FM components

Summary Bats of the speciesNoctilio albiventris, trained to discriminate differences in target distance, emitted pairs of pulses at a rate of 7–10/s, the first a constant frequency (CF) pulse of about 8 ms duration and 75 kHz frequency, followed after about 28 ms by a CF/FM pulse having a 6 ms, 75 kHz CF component that terminates in a 2 ms FM sweep to about 57 kHz.Loud free-running artificial pulses, simulating the bat's natural CF/FM echolocation sound, interfered with distance discrimination at repetition rates exceeding 5/s. Systematic modifications in the temporal and frequency structure of the artificial pulses resulted in orderly changes in the degree of interference. Artificial pulses simulating the natural CF or FM components alone had no effect, nor did 10/s white noise pulses, although constant white noise of the same intensity masked the behavior.Interference occurred when the CF of the artificial pulses was between 52 and 77 kHz, ending with a downward FM sweep of 25 kHz from the CF. For interference to occur there was a much more critical requirement that the FM sweep begin at approximately the frequency of the CF component. The FM sweep needed to be 11 kHz or greater bandwidth. Interference occurred when the duration of the CF component of the CF/FM artificial pulse was between 2 and 30 ms, with maximal effect between 10 and 20 ms. However, a brief (2.0 ms) CF signal 2–27 ms before an isolated FM signal was as effective as a continuous CF component of the same duration.When coupled with the bat's own emissions, artificial CF/FM pulses interfered if they occurred after the bat's CF/FM pulse and before the next natural emission. A 2 ms FM sweep alone was effective in interfering with distance discrimination when it came 8–27 ms after the onset of the bat's own CF/FM pulse. Neither CF/FM nor FM artificial pulses interfered when they began during the bat's own emission. A 10 ms CF pulse alone had no effect at any time.These findings indicate thatN. albiventris uses both the CF and FM components of its short-CF/FM echolocation sound for distance discrimination. The CF onset activates a gating mechanism that, during a narrowly defined subsequent time window, enables the nervous system to process FM pulse-echo pairs for distance information, within a fairly broad frequency range, as long as the frequencies of the CF and the beginning of the FM sweep are nearly identical.Abbreviations CF constant frequency - FM frequency modulation     

Improvement of tactile discrimination performance and enlargement of cortical somatosensory maps after 5 Hz rTMS

Repetitive transcranial magnetic stimulation (rTMS) is increasingly used to investigate mechanisms of brain functions and plasticity, but also as a promising new therapeutic tool. The effects of rTMS depend on the intensity and frequency of stimulation and consist of changes of cortical excitability, which often persists several minutes after termination of rTMS. While these findings imply that cortical processing can be altered by applying current pulses from outside the brain, little is known about how rTMS persistently affects learning and perception. Here we demonstrate in humans, through a combination of psychophysical assessment of two-point discrimination thresholds and functional magnetic resonance imaging (fMRI), that brief periods of 5 Hz rTMS evoke lasting perceptual and cortical changes. rTMS was applied over the cortical representation of the right index finger of primary somatosensory cortex, resulting in a lowering of discrimination thresholds of the right index finger. fMRI revealed an enlargement of the right index finger representation in primary somatosensory cortex that was linearly correlated with the individual rTMS-induced perceptual improvement indicative of a close link between cortical and perceptual changes. The results demonstrate that repetitive, unattended stimulation from outside the brain, combined with a lack of behavioral information, are effective in driving persistent improvement of the perception of touch. The underlying properties and processes that allow cortical networks, after being modified through TMS pulses, to reach new organized stable states that mediate better performance remain to be clarified.     

Rapid and reversible recruitment of early visual cortex for touch

Background

The loss of vision has been associated with enhanced performance in non-visual tasks such as tactile discrimination and sound localization. Current evidence suggests that these functional gains are linked to the recruitment of the occipital visual cortex for non-visual processing, but the neurophysiological mechanisms underlying these crossmodal changes remain uncertain. One possible explanation is that visual deprivation is associated with an unmasking of non-visual input into visual cortex.

Methodology/Principal Findings

We investigated the effect of sudden, complete and prolonged visual deprivation (five days) in normally sighted adult individuals while they were immersed in an intensive tactile training program. Following the five-day period, blindfolded subjects performed better on a Braille character discrimination task. In the blindfold group, serial fMRI scans revealed an increase in BOLD signal within the occipital cortex in response to tactile stimulation after five days of complete visual deprivation. This increase in signal was no longer present 24 hours after blindfold removal. Finally, reversible disruption of occipital cortex function on the fifth day (by repetitive transcranial magnetic stimulation; rTMS) impaired Braille character recognition ability in the blindfold group but not in non-blindfolded controls. This disruptive effect was no longer evident once the blindfold had been removed for 24 hours.

Conclusions/Significance

Overall, our findings suggest that sudden and complete visual deprivation in normally sighted individuals can lead to profound, but rapidly reversible, neuroplastic changes by which the occipital cortex becomes engaged in processing of non-visual information. The speed and dynamic nature of the observed changes suggests that normally inhibited or masked functions in the sighted are revealed by visual loss. The unmasking of pre-existing connections and shifts in connectivity represent rapid, early plastic changes, which presumably can lead, if sustained and reinforced, to slower developing, but more permanent structural changes, such as the establishment of new neural connections in the blind.     

Enhancement of motor coordination by applying high frequency repetitive TMS on the sensory cortex

The sensory function plays an important role for successful motor performance. We investigated the modulating effects of high frequency repetitive transcranial magnetic stimulation (rTMS) on sensory discrimination and motor coordination. Twenty healthy participants were assigned into two random groups; the real- and sham-rTMS group. Total of 900 rTMS pulses at a frequency of 10 Hz (stimulus intensity of 90% RMT) were given over deltoid representational areas of the somatosensory cortex. Sensory discrimination ability was evaluated using two-point discrimination test. Motor coordination was measured by the latency difference between the synchronized contraction of deltoid and abductor pollicis brevis muscles before and after rTMS. The sensory discrimination was significantly increased only in the deltoid area and the difference in the latency of synchronized contraction of two muscles was significantly shortened after real-rTMS compared sham condition, which had tendency of negative correlation following real-rTMS condition. The results of this study demonstrated rTMS-induced enhancement of sensorimotor integration, which may contribute to develop effective therapeutic strategies for rehabilitation of various sensorimotor disorders in the clinical setting.     

Spatial heterogeneity in the perception of face and form attributes

The identity of an object is a fixed property, independent of where it appears, and an effective visual system should capture this invariance [1-3]. However, we now report that the perceived gender of a face is strongly biased toward male or female at different locations in the visual field. The spatial pattern of these biases was distinctive and stable for each individual. Identical neutral faces looked different when they were presented simultaneously at locations maximally biased to opposite genders. A similar effect was observed for perceived age of faces. We measured the magnitude of this perceptual heterogeneity for four other visual judgments: perceived aspect ratio, orientation discrimination, spatial-frequency discrimination, and color discrimination. The effect was sizeable for the aspect ratio task but substantially smaller for the other three tasks. We also evaluated perceptual heterogeneity for facial gender and orientation tasks at different spatial scales. Strong heterogeneity was observed even for the orientation task when tested at small scales. We suggest that perceptual heterogeneity is a general property of visual perception and results from undersampling of the visual signal at spatial scales that are small relative to the size of the receptive fields associated with each visual attribute.     

Inversion leads to quantitative, not qualitative, changes in face processing

Humans are remarkably adept at recognizing objects across a wide range of views. A notable exception to this general rule is that turning a face upside down makes it particularly difficult to recognize. This striking effect has prompted speculation that inversion qualitatively changes the way faces are processed. Researchers commonly assume that configural cues strongly influence the recognition of upright, but not inverted, faces. Indeed, the assumption is so well accepted that the inversion effect itself has been taken as a hallmark of qualitative processing differences. Here, we took a novel approach to understand the inversion effect. We used response classification to obtain a direct view of the perceptual strategies underlying face discrimination and to determine whether orientation effects can be explained by differential contributions of nonlinear processes. Inversion significantly impaired performance in our face discrimination task. However, surprisingly, observers utilized similar, local regions of faces for discrimination in both upright and inverted face conditions, and the relative contributions of nonlinear mechanisms to performance were similar across orientations. Our results suggest that upright and inverted face processing differ quantitatively, not qualitatively; information is extracted more efficiently from upright faces, perhaps as a by-product of orientation-dependent expertise.     

Paired associative stimulation of the auditory system: a proof-of-principle study

Background

Paired associative stimulation (PAS) consisting of repeated application of transcranial magnetic stimulation (TMS) pulses and contingent exteroceptive stimuli has been shown to induce neuroplastic effects in the motor and somatosensory system. The objective was to investigate whether the auditory system can be modulated by PAS.

Methods

Acoustic stimuli (4 kHz) were paired with TMS of the auditory cortex with intervals of either 45 ms (PAS(45 ms)) or 10 ms (PAS(10 ms)). Two-hundred paired stimuli were applied at 0.1 Hz and effects were compared with low frequency repetitive TMS (rTMS) at 0.1 Hz (200 stimuli) and 1 Hz (1000 stimuli) in eleven healthy students. Auditory cortex excitability was measured before and after the interventions by long latency auditory evoked potentials (AEPs) for the tone (4 kHz) used in the pairing, and a control tone (1 kHz) in a within subjects design.

Results

Amplitudes of the N1-P2 complex were reduced for the 4 kHz tone after both PAS(45 ms) and PAS(10 ms), but not after the 0.1 Hz and 1 Hz rTMS protocols with more pronounced effects for PAS(45 ms). Similar, but less pronounced effects were observed for the 1 kHz control tone.

Conclusion

These findings indicate that paired associative stimulation may induce tonotopically specific and also tone unspecific human auditory cortex plasticity.     

Adaptation improves discrimination of face identity

Whether face adaptation confers any advantages to perceptual processing remains an open question. We investigated whether face adaptation can enhance the ability to make fine discriminations in the vicinity of the adapted face. We compared face discrimination thresholds in three adapting conditions: (i) same-face: where adapting and test faces were the same, (ii) different-face: where adapting and test faces differed, and (iii) baseline: where the adapting stimulus was a blank. Discrimination thresholds for morphed identity changes involving the adapted face (same-face) improved compared with those from both the baseline (no-adaptation) and different-face conditions. Since adapting to a face did not alter discrimination performance for other faces, this effect is selective for the facial identity that is adapted. These results indicate a form of gain control to heighten perceptual sensitivity in the vicinity of a currently viewed face, analogous to forms of adaptive gain control at lower levels of the visual system.