Stable face representations

Photographs are often used to establish the identity of an individual or to verify that they are who they claim to be. Yet, recent research shows that it is surprisingly difficult to match a photo to a face. Neither humans nor machines can perform this task reliably. Although human perceivers are good at matching familiar faces, performance with unfamiliar faces is strikingly poor. The situation is no better for automatic face recognition systems. In practical settings, automatic systems have been consistently disappointing. In this review, we suggest that failure to distinguish between familiar and unfamiliar face processing has led to unrealistic expectations about face identification in applied settings. We also argue that a photograph is not necessarily a reliable indicator of facial appearance, and develop our proposal that summary statistics can provide more stable face representations. In particular, we show that image averaging stabilizes facial appearance by diluting aspects of the image that vary between snapshots of the same person. We review evidence that the resulting images can outperform photographs in both behavioural experiments and computer simulations, and outline promising directions for future research.     

The many faces of research on face perception

Face perception is fundamental to human social interaction. Many different types of important information are visible in faces and the processes and mechanisms involved in extracting this information are complex and can be highly specialized. The importance of faces has long been recognized by a wide range of scientists. Importantly, the range of perspectives and techniques that this breadth has brought to face perception research has, in recent years, led to many important advances in our understanding of face processing. The articles in this issue on face perception each review a particular arena of interest in face perception, variously focusing on (i) the social aspects of face perception (attraction, recognition and emotion), (ii) the neural mechanisms underlying face perception (using brain scanning, patient data, direct stimulation of the brain, visual adaptation and single-cell recording), and (iii) comparative aspects of face perception (comparing adult human abilities with those of chimpanzees and children). Here, we introduce the central themes of the issue and present an overview of the articles.     

Inversion leads to quantitative, not qualitative, changes in face processing

Humans are remarkably adept at recognizing objects across a wide range of views. A notable exception to this general rule is that turning a face upside down makes it particularly difficult to recognize. This striking effect has prompted speculation that inversion qualitatively changes the way faces are processed. Researchers commonly assume that configural cues strongly influence the recognition of upright, but not inverted, faces. Indeed, the assumption is so well accepted that the inversion effect itself has been taken as a hallmark of qualitative processing differences. Here, we took a novel approach to understand the inversion effect. We used response classification to obtain a direct view of the perceptual strategies underlying face discrimination and to determine whether orientation effects can be explained by differential contributions of nonlinear processes. Inversion significantly impaired performance in our face discrimination task. However, surprisingly, observers utilized similar, local regions of faces for discrimination in both upright and inverted face conditions, and the relative contributions of nonlinear mechanisms to performance were similar across orientations. Our results suggest that upright and inverted face processing differ quantitatively, not qualitatively; information is extracted more efficiently from upright faces, perhaps as a by-product of orientation-dependent expertise.     

Abnormal FMRI adaptation to unfamiliar faces in a case of developmental prosopamnesia

In rare cases, damage to the temporal lobe causes a selective impairment in the ability to learn new faces, a condition known as prosopamnesia [1]. Here we present the case of an individual with prosopamnesia in the absence of any acquired structural lesion. "C" shows intact processing of simple and complex nonface objects, but her ability to learn new faces is severely impaired. We used a neural marker of perceptual learning known as repetition suppression to examine functioning within C's fusiform face area (FFA), a region of cortex involved in face perception [2]. For comparison, we examined repetition suppression in the scene-selective parahippocampal place area (PPA) [3]. As expected, normal controls showed significant region-specific attenuation of neural activity across repetitions of each stimulus class. C also showed normal attenuation within the PPA to familiar and unfamiliar scenes, and within the FFA to familiar faces. Critically, however, she failed to show any adaptive change within the FFA for repeated unfamiliar faces, despite a face-specific blood-oxygen-dependent response (BOLD) response in her FFA during viewing of face stimuli. Our findings suggest that in developmental prosopamnesia, the FFA cannot maintain stable representations of new faces for subsequent recall or recognition.     

Neural correlates of perceiving emotional faces and bodies in developmental prosopagnosia: an event-related fMRI-study

Many people experience transient difficulties in recognizing faces but only a small number of them cannot recognize their family members when meeting them unexpectedly. Such face blindness is associated with serious problems in everyday life. A better understanding of the neuro-functional basis of impaired face recognition may be achieved by a careful comparison with an equally unique object category and by a adding a more realistic setting involving neutral faces as well facial expressions. We used event-related functional magnetic resonance imaging (fMRI) to investigate the neuro-functional basis of perceiving faces and bodies in three developmental prosopagnosics (DP) and matched healthy controls. Our approach involved materials consisting of neutral faces and bodies as well as faces and bodies expressing fear or happiness. The first main result is that the presence of emotional information has a different effect in the patient vs. the control group in the fusiform face area (FFA). Neutral faces trigger lower activation in the DP group, compared to the control group, while activation for facial expressions is the same in both groups. The second main result is that compared to controls, DPs have increased activation for bodies in the inferior occipital gyrus (IOG) and for neutral faces in the extrastriate body area (EBA), indicating that body and face sensitive processes are less categorically segregated in DP. Taken together our study shows the importance of using naturalistic emotional stimuli for a better understanding of developmental face deficits.     

The effect of face inversion on activity in human neural systems for face and object perception

The differential effect of stimulus inversion on face and object recognition suggests that inverted faces are processed by mechanisms for the perception of other objects rather than by face perception mechanisms. We investigated the face inversion using functional magnetic resonance imaging (fMRI). The principal effect of face inversion on was an increased response in ventral extrastriate regions that respond preferentially to another class of objects (houses). In contrast, house inversion did not produce a similar change in face-selective regions. Moreover, stimulus inversion had equivalent, minimal effects for faces in in face-selective regions and for houses in house-selective regions. The results suggest that the failure of face perception systems with inverted faces leads to the recruitment of processing resources in object perception systems, but this failure is not reflected by altered activity in face perception systems.     

Voice cells in the primate temporal lobe

Communication signals are important for social interactions and survival and are thought to receive specialized processing in the visual and auditory systems. Whereas the neural processing of faces by face clusters and face cells has been repeatedly studied [1-5], less is known about the neural representation of voice content. Recent functional magnetic resonance imaging (fMRI) studies have localized voice-preferring regions in the primate temporal lobe [6, 7], but the hemodynamic response cannot directly assess neurophysiological properties. We investigated the responses of neurons in an fMRI-identified voice cluster in awake monkeys, and here we provide the first systematic evidence for voice cells. "Voice cells" were identified, in analogy to "face cells," as neurons responding at least 2-fold stronger to conspecific voices than to "nonvoice" sounds or heterospecific voices. Importantly, whereas face clusters are thought to contain high proportions of face cells [4] responding broadly to many faces [1, 2, 4, 5, 8-10], we found that voice clusters contain moderate proportions of voice cells. Furthermore, individual voice cells exhibit high stimulus selectivity. The results reveal the neurophysiological bases for fMRI-defined voice clusters in the primate brain and highlight potential differences in how the auditory and?visual systems generate selective representations of communication signals.     

Cross-Category Adaptation: Objects Produce Gender Adaptation in the Perception of Faces

Adaptation aftereffects have been found for low-level visual features such as colour, motion and shape perception, as well as higher-level features such as gender, race and identity in domains such as faces and biological motion. It is not yet clear if adaptation effects in humans extend beyond this set of higher order features. The aim of this study was to investigate whether objects highly associated with one gender, e.g. high heels for females or electric shavers for males can modulate gender perception of a face. In two separate experiments, we adapted subjects to a series of objects highly associated with one gender and subsequently asked participants to judge the gender of an ambiguous face. Results showed that participants are more likely to perceive an ambiguous face as male after being exposed to objects highly associated to females and vice versa. A gender adaptation aftereffect was obtained despite the adaptor and test stimuli being from different global categories (objects and faces respectively). These findings show that our perception of gender from faces is highly affected by our environment and recent experience. This suggests two possible mechanisms: (a) that perception of the gender associated with an object shares at least some brain areas with those responsible for gender perception of faces and (b) adaptation to gender, which is a high-level concept, can modulate brain areas that are involved in facial gender perception through top-down processes.     

Impairments of Biological Motion Perception in Congenital Prosopagnosia

Prosopagnosia is a deficit in recognizing people from their faces. Acquired prosopagnosia results after brain damage, developmental or congenital prosopagnosia (CP) is not caused by brain lesion, but has presumably been present from early childhood onwards. Since other sensory, perceptual, and cognitive abilities are largely spared, CP is considered to be a stimulus-specific deficit, limited to face processing. Given that recent behavioral and imaging studies indicate a close relationship of face and biological-motion perception in healthy adults, we hypothesized that biological motion processing should be impaired in CP. Five individuals with CP and ten matched healthy controls were tested with diverse biological-motion stimuli and tasks. Four of the CP individuals showed severe deficits in biological-motion processing, while one performed within the lower range of the controls. A discriminant analysis classified all participants correctly with a very high probability for each participant. These findings demonstrate that in CP, impaired perception of faces can be accompanied by impaired biological-motion perception. We discuss implications for dedicated and shared mechanisms involved in the perception of faces and biological motion.     

Recognizing faces

Face recognition is an area where research has increased considerably in recent years, yet theoretical progress has been slow. Here it is argued that by considering the perception and recognition of familiar faces, as well as episodic memory for unfamiliar faces, a functional framework for face recognition can be developed. Experiments using faces, that include tasks analogous to 'visual search' and 'lexical decision' are described, and the processes in operation are compared with those occurring in word recognition. The results allow us to distinguish a number of possible subcomponents for a functional model of face recognition.     

Electrical Stimulation over Bilateral Occipito-Temporal Regions Reduces N170 in the Right Hemisphere and the Composite Face Effect

Transcranial direct current stimulation (tDCS) is a non-invasive brain stimulation technique that can modulate cortical excitability. Although the clinical value of tDCS has been advocated, the potential of tDCS in cognitive rehabilitation of face processing deficits is less understood. Face processing has been associated with the occipito-temporal cortex (OT). The present study investigated whether face processing in healthy adults can be modulated by applying tDCS over the OT. Experiment 1 investigated whether tDCS can affect N170, a face-sensitive ERP component, with a face orientation judgment task. The N170 in the right hemisphere was reduced in active stimulation conditions compared with the sham stimulation condition for both upright faces and inverted faces. Experiment 2 further demonstrated that tDCS can modulate the composite face effect, a type of holistic processing that reflects the obligatory attention to all parts of a face. The composite face effect was reduced in active stimulation conditions compared with the sham stimulation condition. Additionally, the current polarity did not modulate the effect of tDCS in the two experiments. The present study demonstrates that N170 can be causally manipulated by stimulating the OT with weak currents. Furthermore, our study provides evidence that obligatory attention to all parts of a face can be affected by the commonly used tDCS parameter setting.     

Spatial and Temporal Attention Modulate the Early Stages of Face Processing: Behavioural Evidence from a Reaching Paradigm

A presently unresolved question within the face perception literature is whether attending to the location of a face modulates face processing (i.e. spatial attention). Opinions on this matter diverge along methodological lines – where neuroimaging studies have observed that the allocation of spatial attention serves to enhance the neural response to a face, findings from behavioural paradigms suggest face processing is carried out independently of spatial attention. In the present study, we reconcile this divide by using a continuous behavioural response measure that indexes face processing at a temporal resolution not available in discrete behavioural measures (e.g. button press). Using reaching trajectories as our response measure, we observed that although participants were able to process faces both when attended and unattended (as others have found), face processing was not impervious to attentional modulation. Attending to the face conferred clear benefits on sex-classification processes at less than 350ms of stimulus processing time. These findings constitute the first reliable demonstration of the modulatory effects of both spatial and temporal attention on face processing within a behavioural paradigm.     

Reverse Correlating Love: Highly Passionate Women Idealize Their Partner’s Facial Appearance

A defining feature of passionate love is idealization—evaluating romantic partners in an overly favorable light. Although passionate love can be expected to color how favorably individuals represent their partner in their mind, little is known about how passionate love is linked with visual representations of the partner. Using reverse correlation techniques for the first time to study partner representations, the present study investigated whether women who are passionately in love represent their partner’s facial appearance more favorably than individuals who are less passionately in love. In a within-participants design, heterosexual women completed two forced-choice classification tasks, one for their romantic partner and one for a male acquaintance, and a measure of passionate love. In each classification task, participants saw two faces superimposed with noise and selected the face that most resembled their partner (or an acquaintance). Classification images for each of high passion and low passion groups were calculated by averaging across noise patterns selected as resembling the partner or the acquaintance and superimposing the averaged noise on an average male face. A separate group of women evaluated the classification images on attractiveness, trustworthiness, and competence. Results showed that women who feel high (vs. low) passionate love toward their partner tend to represent his face as more attractive and trustworthy, even when controlling for familiarity effects using the acquaintance representation. Using an innovative method to study partner representations, these findings extend our understanding of cognitive processes in romantic relationships.     

The fusiform face area: a cortical region specialized for the perception of faces

Faces are among the most important visual stimuli we perceive, informing us not only about a person's identity, but also about their mood, sex, age and direction of gaze. The ability to extract this information within a fraction of a second of viewing a face is important for normal social interactions and has probably played a critical role in the survival of our primate ancestors. Considerable evidence from behavioural, neuropsychological and neurophysiological investigations supports the hypothesis that humans have specialized cognitive and neural mechanisms dedicated to the perception of faces (the face-specificity hypothesis). Here, we review the literature on a region of the human brain that appears to play a key role in face perception, known as the fusiform face area (FFA). Section 1 outlines the theoretical background for much of this work. The face-specificity hypothesis falls squarely on one side of a longstanding debate in the fields of cognitive science and cognitive neuroscience concerning the extent to which the mind/brain is composed of: (i) special-purpose ('domain-specific') mechanisms, each dedicated to processing a specific kind of information (e.g. faces, according to the face-specificity hypothesis), versus (ii) general-purpose ('domain-general') mechanisms, each capable of operating on any kind of information. Face perception has long served both as one of the prime candidates of a domain-specific process and as a key target for attack by proponents of domain-general theories of brain and mind. Section 2 briefly reviews the prior literature on face perception from behaviour and neurophysiology. This work supports the face-specificity hypothesis and argues against its domain-general alternatives (the individuation hypothesis, the expertise hypothesis and others). Section 3 outlines the more recent evidence on this debate from brain imaging, focusing particularly on the FFA. We review the evidence that the FFA is selectively engaged in face perception, by addressing (and rebutting) five of the most widely discussed alternatives to this hypothesis. In section 4, we consider recent findings that are beginning to provide clues into the computations conducted in the FFA and the nature of the representations the FFA extracts from faces. We argue that the FFA is engaged both in detecting faces and in extracting the necessary perceptual information to recognize them, and that the properties of the FFA mirror previously identified behavioural signatures of face-specific processing (e.g. the face-inversion effect). Section 5 asks how the computations and representations in the FFA differ from those occurring in other nearby regions of cortex that respond strongly to faces and objects. The evidence indicates clear functional dissociations between these regions, demonstrating that the FFA shows not only functional specificity but also area specificity. We end by speculating in section 6 on some of the broader questions raised by current research on the FFA, including the developmental origins of this region and the question of whether faces are unique versus whether similarly specialized mechanisms also exist for other domains of high-level perception and cognition.     

Neurophysiological mechanisms underlying face processing within and beyond the temporal cortical visual areas.

The ways in which information about faces is represented and stored in the temporal lobe visual areas of primates, as shown by recordings from single neurons in macaques, are considered. Some neurons that respond primarily to faces are found in the cortex in the anterior part of the superior temporal sulcus (in which neurons are especially likely to be tuned to facial expression and to face movement involved in gesture), and in the TE areas more ventrally forming the inferior temporal gyrus (in which neurons are more likely to have responses related to the identity of faces). Quantitative studies of the responses of the neurons that respond differently to the faces of different individuals show that information about the identity of the individual is represented by the responses of a population of neurons, that is, ensemble encoding rather than 'grandmother cell' encoding is used. It is argued that this type of tuning is a delicate compromise between very fine tuning, which has the advantage of low interference in neuronal network operations but the disadvantage of losing the useful properties (such as generalization, completion and graceful degradation) of storage in neuronal networks, and broad tuning, which has the advantage of allowing these properties of neuronal networks to be realized but the disadvantage of leading to interference between the different memories stored in an associative network. There is evidence that the responses of some of these neurons are altered by experience so that new stimuli become incorporated in the network. It is shown that the representation that is built in temporal cortical areas shows considerable invariance for size, contrast, spatial frequency and translation. Thus the representation is in a form which is particularly useful for storage and as an output from the visual system. It is also shown that one of the representations that is built is object based, which is suitable for recognition and as an input to associative memory, and that another is viewer centred, which is appropriate for conveying information about gesture. Ways are considered in which such cortical representations might be built by competitive self-organization aided by back projections in the multi-stage cortical processing hierarchy which has convergence from stage to stage.     

Using Effort to Measure Reward Value of Faces in Children with Autism

According to one influential account, face processing atypicalities in autism reflect reduced reward value of faces, which results in limited attention to faces during development and a consequent failure to acquire face expertise. Surprisingly, however, there is a paucity of work directly investigating the reward value of faces for individuals with autism and the evidence for diminished face rewards in this population remains equivocal. In the current study, we measured how hard children with autism would work to view faces, using an effortful key-press sequence, and whether they were sensitive to the differential reward value of attractive and unattractive faces. Contrary to expectations, cognitively able children with autism did not differ from typically developing children of similar age and ability in their willingness to work to view faces. Moreover, the effort expended was strongly positively correlated with facial attractiveness ratings in both groups of children. There was also no evidence of atypical reward values for other, less social categories (cars and inverted faces) in the children with autism. These results speak against the possibility that face recognition difficulties in autism are explained by atypical reward value of faces.     

The role of gamma-band activity in the representation of faces: reduced activity in the fusiform face area in congenital prosopagnosia

Background

Congenital prosopagnosia (CP) describes an impairment in face processing that is presumably present from birth. The neuronal correlates of this dysfunction are still under debate. In the current paper, we investigate high-frequent oscillatory activity in response to faces in persons with CP. Such neuronal activity is thought to reflect higher-level representations for faces.

Methodology

Source localization of induced Gamma-Band Responses (iGBR) measured by magnetoencephalography (MEG) was used to establish the origin of oscillatory activity in response to famous and unknown faces which were presented in upright and inverted orientation. Persons suffering from congenital prosopagnosia (CP) were compared to matched controls.

Principal Findings

Corroborating earlier research, both groups revealed amplified iGBR in response to upright compared to inverted faces predominately in a time interval between 170 and 330 ms and in a frequency range from 50–100 Hz. Oscillatory activity upon known faces was smaller in comparison to unknown faces, suggesting a “sharpening” effect reflecting more efficient processing for familiar stimuli. These effects were seen in a wide cortical network encompassing temporal and parietal areas involved in the disambiguation of homogenous stimuli such as faces, and in the retrieval of semantic information. Importantly, participants suffering from CP displayed a strongly reduced iGBR in the left fusiform area compared to control participants.

Conclusions

In sum, these data stress the crucial role of oscillatory activity for face representation and demonstrate the involvement of a distributed occipito-temporo-parietal network in generating iGBR. This study also provides the first evidence that persons suffering from an agnosia actually display reduced gamma band activity. Finally, the results argue strongly against the view that oscillatory activity is a mere epiphenomenon brought fourth by rapid eye-movements (micro saccades).     

Second-order relational manipulations affect both humans and monkeys

Recognition and individuation of conspecifics by their face is essential for primate social cognition. This ability is driven by a mechanism that integrates the appearance of facial features with subtle variations in their configuration (i.e., second-order relational properties) into a holistic representation. So far, there is little evidence of whether our evolutionary ancestors show sensitivity to featural spatial relations and hence holistic processing of faces as shown in humans. Here, we directly compared macaques with humans in their sensitivity to configurally altered faces in upright and inverted orientations using a habituation paradigm and eye tracking technologies. In addition, we tested for differences in processing of conspecific faces (human faces for humans, macaque faces for macaques) and non-conspecific faces, addressing aspects of perceptual expertise. In both species, we found sensitivity to second-order relational properties for conspecific (expert) faces, when presented in upright, not in inverted, orientation. This shows that macaques possess the requirements for holistic processing, and thus show similar face processing to that of humans.     

面孔分类中空间高低频表征的神经机制：一个颅内脑电研究

来自多方面的研究表明,面孔的分类和识别位于特定脑区．同时,已有行为实验研究表明,图像的空间高低频特征在面孔分类的不同范畴中起不同的贡献,例如身份更多被低频信号传递,性别被高低频共同传递,而表情更多被高频传递．然而,空间频率在面孔分类中的贡献,其表征和神经机制目前相关研究很少．利用特定癫痫患者植入颅内电极的监控期,呈现不同类型面孔图像,同时记录其颅内脑电,用事件相关电位方法考察了据认为是面孔特定成分的相关电位的潜伏期在170 ms的波形(N170波形)的变化;用电极反应显著性分析考察了空间频率在不同分类特征上的贡献．结果表明,空间高频(HSF)图像的N170潜伏期显著延迟．只呈现空间低频(LSF)图像,N170的潜伏期对普通人面孔会延迟,而对熟悉的名人则没有这个差异．女性面孔诱发的N170在HSF条件下潜伏期明显晚于LSF条件,而男性面孔诱发的波形则不存在这个差异．表情在N170上没有体现出任何差异．但是基于电极的显著性分析表明,有更多的额叶电极参与了表情的加工;身份特征加工有更多电极在空间低频上表现出差异,而性别加工则空间高低频比较平衡．与以往行为结果不同的是,表情加工也有更多低频贡献,而且表情的差异可以在早达114 ms的时候就发生．这符合表情信息在颞枕区域有一个快速基本加工,再传递到其他脑区的认知模型．因此,空间高低频信息在身份和性别上的贡献,可能发生在经典的面孔加工脑区,由N170表达,表情信息不由N170表达,而是在颞枕较广泛的范围内快速加工再传递到别的脑区,如额叶．这是首次利用颅内脑电就空间频率在面孔分类中的贡献的神经机制进行研究,为深入理解脑内面孔各种特征加工的动态过程提供了一个新的切入点．     

Attachment and Children’s Biased Attentional Processing: Evidence for the Exclusion of Attachment-Related Information

Research in both infants and adults demonstrated that attachment expectations are associated with the attentional processing of attachment-related information. However, this research suffered from methodological issues and has not been validated across ages. Employing a more ecologically valid paradigm to measure attentional processes by virtue of eye tracking, the current study tested the defensive exclusion hypothesis in late childhood. According to this hypothesis, insecurely attached children are assumed to defensively exclude attachment-related information. We hypothesized that securely attached children process attachment- related neutral and emotional information in a more open manner compared to insecurely attached children. Sixty-two children (59.7% girls, 8–12 years) completed two different tasks, while eye movements were recorded: task one presented an array of neutral faces including mother and unfamiliar women and task two presented the same with happy and angry faces. Results indicated that more securely attached children looked longer at mother’s face regardless of the emotional expression. Also, they tend to have more maintained attention to mother’s neutral face. Furthermore, more attachment avoidance was related to a reduced total viewing time of mother’s neutral, happy, and angry face. Attachment anxiety was not consistently related to the processing of mother’s face. Findings support the theoretical assumption that securely attached children have an open manner of processing all attachment-related information.