Identification and classification of facial familiarity in directed lying: an ERP study

Recognizing familiar faces is essential to social functioning, but little is known about how people identify human faces and classify them in terms of familiarity. Face identification involves discriminating familiar faces from unfamiliar faces, whereas face classification involves making an intentional decision to classify faces as "familiar" or "unfamiliar." This study used a directed-lying task to explore the differentiation between identification and classification processes involved in the recognition of familiar faces. To explore this issue, the participants in this study were shown familiar and unfamiliar faces. They responded to these faces (i.e., as familiar or unfamiliar) in accordance with the instructions they were given (i.e., to lie or to tell the truth) while their EEG activity was recorded. Familiar faces (regardless of lying vs. truth) elicited significantly less negative-going N400f in the middle and right parietal and temporal regions than unfamiliar faces. Regardless of their actual familiarity, the faces that the participants classified as "familiar" elicited more negative-going N400f in the central and right temporal regions than those classified as "unfamiliar." The P600 was related primarily with the facial identification process. Familiar faces (regardless of lying vs. truth) elicited more positive-going P600f in the middle parietal and middle occipital regions. The results suggest that N400f and P600f play different roles in the processes involved in facial recognition. The N400f appears to be associated with both the identification (judgment of familiarity) and classification of faces, while it is likely that the P600f is only associated with the identification process (recollection of facial information). Future studies should use different experimental paradigms to validate the generalizability of the results of this study.     

Effects of Caricaturing in Shape or Color on Familiarity Decisions for Familiar and Unfamiliar Faces

Recent evidence suggests that while reflectance information (including color) may be more diagnostic for familiar face recognition, shape may be more diagnostic for unfamiliar face identity processing. Moreover, event-related potential (ERP) findings suggest an earlier onset for neural processing of facial shape compared to reflectance. In the current study, we aimed to explore specifically the roles of facial shape and color in a familiarity decision task using pre-experimentally familiar (famous) and unfamiliar faces that were caricatured either in shape-only, color-only, or both (full; shape + color) by 15%, 30%, or 45%. We recorded accuracies, mean reaction times, and face-sensitive ERPs. Performance data revealed that shape caricaturing facilitated identity processing for unfamiliar faces only. In the ERP data, such effects of shape caricaturing emerged earlier than those of color caricaturing. Unsurprisingly, ERP effects were accentuated for larger levels of caricaturing. Overall, our findings corroborate the importance of shape for identity processing of unfamiliar faces and demonstrate an earlier onset of neural processing for facial shape compared to color.     

Stable face representations

Photographs are often used to establish the identity of an individual or to verify that they are who they claim to be. Yet, recent research shows that it is surprisingly difficult to match a photo to a face. Neither humans nor machines can perform this task reliably. Although human perceivers are good at matching familiar faces, performance with unfamiliar faces is strikingly poor. The situation is no better for automatic face recognition systems. In practical settings, automatic systems have been consistently disappointing. In this review, we suggest that failure to distinguish between familiar and unfamiliar face processing has led to unrealistic expectations about face identification in applied settings. We also argue that a photograph is not necessarily a reliable indicator of facial appearance, and develop our proposal that summary statistics can provide more stable face representations. In particular, we show that image averaging stabilizes facial appearance by diluting aspects of the image that vary between snapshots of the same person. We review evidence that the resulting images can outperform photographs in both behavioural experiments and computer simulations, and outline promising directions for future research.     

Familiar Face Detection in 180ms

The visual system is tuned for rapid detection of faces, with the fastest choice saccade to a face at 100ms. Familiar faces have a more robust representation than do unfamiliar faces, and are detected faster in the absence of awareness and with reduced attentional resources. Faces of family and close friends become familiar over a protracted period involving learning the unique visual appearance, including a view-invariant representation, as well as person knowledge. We investigated the effect of personal familiarity on the earliest stages of face processing by using a saccadic-choice task to measure how fast familiar face detection can happen. Subjects made correct and reliable saccades to familiar faces when unfamiliar faces were distractors at 180ms—very rapid saccades that are 30 to 70ms earlier than the earliest evoked potential modulated by familiarity. By contrast, accuracy of saccades to unfamiliar faces with familiar faces as distractors did not exceed chance. Saccades to faces with object distractors were even faster (110 to 120 ms) and equivalent for familiar and unfamiliar faces, indicating that familiarity does not affect ultra-rapid saccades. We propose that detectors of diagnostic facial features for familiar faces develop in visual cortices through learning and allow rapid detection that precedes explicit recognition of identity.     

Facial Recognition in a Group-Living Cichlid Fish

The theoretical underpinnings of the mechanisms of sociality, e.g. territoriality, hierarchy, and reciprocity, are based on assumptions of individual recognition. While behavioural evidence suggests individual recognition is widespread, the cues that animals use to recognise individuals are established in only a handful of systems. Here, we use digital models to demonstrate that facial features are the visual cue used for individual recognition in the social fish Neolamprologus pulcher. Focal fish were exposed to digital images showing four different combinations of familiar and unfamiliar face and body colorations. Focal fish attended to digital models with unfamiliar faces longer and from a further distance to the model than to models with familiar faces. These results strongly suggest that fish can distinguish individuals accurately using facial colour patterns. Our observations also suggest that fish are able to rapidly (≤ 0.5 sec) discriminate between familiar and unfamiliar individuals, a speed of recognition comparable to primates including humans.     

Neurophysiological mechanisms underlying face processing within and beyond the temporal cortical visual areas.

The ways in which information about faces is represented and stored in the temporal lobe visual areas of primates, as shown by recordings from single neurons in macaques, are considered. Some neurons that respond primarily to faces are found in the cortex in the anterior part of the superior temporal sulcus (in which neurons are especially likely to be tuned to facial expression and to face movement involved in gesture), and in the TE areas more ventrally forming the inferior temporal gyrus (in which neurons are more likely to have responses related to the identity of faces). Quantitative studies of the responses of the neurons that respond differently to the faces of different individuals show that information about the identity of the individual is represented by the responses of a population of neurons, that is, ensemble encoding rather than 'grandmother cell' encoding is used. It is argued that this type of tuning is a delicate compromise between very fine tuning, which has the advantage of low interference in neuronal network operations but the disadvantage of losing the useful properties (such as generalization, completion and graceful degradation) of storage in neuronal networks, and broad tuning, which has the advantage of allowing these properties of neuronal networks to be realized but the disadvantage of leading to interference between the different memories stored in an associative network. There is evidence that the responses of some of these neurons are altered by experience so that new stimuli become incorporated in the network. It is shown that the representation that is built in temporal cortical areas shows considerable invariance for size, contrast, spatial frequency and translation. Thus the representation is in a form which is particularly useful for storage and as an output from the visual system. It is also shown that one of the representations that is built is object based, which is suitable for recognition and as an input to associative memory, and that another is viewer centred, which is appropriate for conveying information about gesture. Ways are considered in which such cortical representations might be built by competitive self-organization aided by back projections in the multi-stage cortical processing hierarchy which has convergence from stage to stage.     

Prioritized Detection of Personally Familiar Faces

We investigated whether personally familiar faces are preferentially processed in conditions of reduced attentional resources and in the absence of conscious awareness. In the first experiment, we used Rapid Serial Visual Presentation (RSVP) to test the susceptibility of familiar faces and faces of strangers to the attentional blink. In the second experiment, we used continuous flash interocular suppression to render stimuli invisible and measured face detection time for personally familiar faces as compared to faces of strangers. In both experiments we found an advantage for detection of personally familiar faces as compared to faces of strangers. Our data suggest that the identity of faces is processed with reduced attentional resources and even in the absence of awareness. Our results show that this facilitated processing of familiar faces cannot be attributed to detection of low-level visual features and that a learned unique configuration of facial features can influence preconscious perceptual processing.     

Dynamics of population code for working memory in the prefrontal cortex

Some neurons (delay cells) in the prefrontal cortex elevate their activities throughout the time period during which the animal is required to remember past events and prepare future behavior, suggesting that working memory is mediated by continuous neural activity. It is unknown, however, how working memory is represented within a population of prefrontal cortical neurons. We recorded from neuronal ensembles in the prefrontal cortex as rats learned a new delayed alternation task. Ensemble activities changed in parallel with behavioral learning so that they increasingly allowed correct decoding of previous and future goal choices. In well-trained rats, considerable decoding was possible based on only a few neurons and after removing continuously active delay cells. These results show that neural activity in the prefrontal cortex changes dynamically during new task learning so that working memory is robustly represented and that working memory can be mediated by sequential activation of different neural populations.     

Spontaneous gender categorization in masking and priming studies: key for distinguishing Jane from John Doe but not Madonna from Sinatra

Facial recognition is key to social interaction, however with unfamiliar faces only generic information, in the form of facial stereotypes such as gender and age is available. Therefore is generic information more prominent in unfamiliar versus familiar face processing? In order to address the question we tapped into two relatively disparate stages of face processing. At the early stages of encoding, we employed perceptual masking to reveal that only perception of unfamiliar face targets is affected by the gender of the facial masks. At the semantic end; using a priming paradigm, we found that while to-be-ignored unfamiliar faces prime lexical decisions to gender congruent stereotypic words, familiar faces do not. Our findings indicate that gender is a more salient dimension in unfamiliar relative to familiar face processing, both in early perceptual stages as well as later semantic stages of person construal.     

Visual discrimination of primate species based on faces in chimpanzees

Many primate studies have investigated discrimination of individual faces within the same species. However, few studies have looked at discrimination between primate species faces at the categorical level. This study systematically examined the factors important for visual discrimination between primate species faces in chimpanzees, including: colour, orientation, familiarity, and perceptual similarity. Five adult female chimpanzees were tested on their ability to discriminate identical and categorical (non-identical) images of different primate species faces in a series of touchscreen matching-to-sample experiments. Discrimination performance for chimpanzee, gorilla, and orangutan faces was better in colour than in greyscale. An inversion effect was also found, with higher accuracy for upright than inverted faces. Discrimination performance for unfamiliar (baboon and capuchin monkey) and highly familiar (chimpanzee and human) but perceptually different species was equally high. After excluding effects of colour and familiarity, difficulty in discriminating between different species faces can be best explained by their perceptual similarity to each other. Categorical discrimination performance for unfamiliar, perceptually similar faces (gorilla and orangutan) was significantly worse than unfamiliar, perceptually different faces (baboon and capuchin monkey). Moreover, multidimensional scaling analysis of the image similarity data based on local feature matching revealed greater similarity between chimpanzee, gorilla and orangutan faces than between human, baboon and capuchin monkey faces. We conclude our chimpanzees appear to perceive similarity in primate faces in a similar way to humans. Information about perceptual similarity is likely prioritized over the potential influence of previous experience or a conceptual representation of species for categorical discrimination between species faces.     

The Effect of Familiarity on Behavior of Kenneled Dogs During Interactions With Conspecifics

Kenneled environments often prevent direct physical contact between dogs, potentially causing stress, and so it has been recommended that such contact should be provided. This study examined the effect of familiarity on the behavior of dogs during off-lead interaction. Kenneled dogs (3 breeds) were given 15-min off-lead interactions with a familiar dog and an unfamiliar dog; the behavior of the focal dog and the distance between the dogs were recorded. More time in contact and interaction behaviors were recorded at 0 to 3 min with unfamiliar dogs than with familiar dogs. At 9 to 12 min, familiar pairs spent more time within 5 body lengths and more time being followed than unfamiliar pairs, who spent more time at more than 5 body lengths apart. This suggests that the initial interaction is more important when dogs are unfamiliar, but once this “greeting” has occurred, unfamiliar pairs are more likely to investigate their surroundings independently rather than together. Breed differences were observed only at 0 to 3 min. The study suggests that familiarity should be taken into account when assessing the effectiveness of conspecific contact as a potential enrichment for kennel-housed dogs.     

Orangutans (Pongo pygmaeus) Remember Old Acquaintances

Many social animals can discriminate between familiar and unfamiliar faces. Orangutans, however, lead a semi-solitary life and spend much of the day alone. As such, they may be less adept at recognizing conspecifics and are a good model for determining how social structure influences the evolution of social cognition such as facial recognition. The present study is the first report of whether orangutans can distinguish among individual faces. We adopted a preferential looking method and found that orangutans used facial discrimination to identify known conspecifics. This suggests that frequent and intense social interaction is not necessary for facial discrimination, although our findings were limited by the small number of stimuli and the unequal numbers of male and female orangutans depicted in the stimuli.     

Patterns of spatio-temporal correlations in the neural activity of the cat motor cortex during trained forelimb movements

In order to study how neurons in the primary motor cortex (MI) are dynamically linked together during skilled movement, we recorded simultaneously from many cortical neurons in cats trained to perform a reaching and retrieval task using their forelimbs. Analysis of task-related spike activity in the MI of the hemisphere contralateral to the reaching forelimb (in identified forelimb or hindlimb representations) recorded through chronically implanted microwires, was followed by pairwise evaluation of temporally correlated activity in these neurons during task performance using shuffle corrected cross-correlograms. Over many months of recording, a variety of task-related modulations of neural activities were observed in individual efferent zones. Positively correlated activity (mainly narrow peaks at zero or short latencies) was seen during task performance frequently between neurons recorded within the forelimb representation of MI, rarely within the hindlimb area of MI, and never between forelimb and hindlimb areas. Correlated activity was frequently observed between neurons with different patterns of task-related activity or preferential activity during different task elements (reaching, feeding, etc.), and located in efferent zones with dissimilar representation as defined by intracortical microstimulation. The observed synchronization of action potentials among selected but functionally varied groups of MI neurons possibly reflects dynamic recruitment of network connections between efferent zones during skilled movement.     

Neuronal population activity and functional imaging.

Human functional brain imaging detects blood flow changes that are thought to reflect the activity of neuronal populations and, thus, the responses of neurons that carry behaviourally relevant information. Since this relationship is poorly understood, we explored the link between the activity of single neurons and their neuronal population. The functional imaging results were in good agreement with levels of population activation predicted from the known effects of sensory stimulation, learning and attention on single cortical neurons. However, the nature of the relationship between population activation and single neuron firing was very surprising. Population activation was strongly influenced by those neurons firing at low rates and so was very sensitive to the baseline or 'spontaneous' firing rate. When neural representations were sparse and neurons were tuned to several stimulus dimensions, population activation was hardly influenced by the few neurons whose firing was most strongly modulated by the task or stimulus. Measures of population activation could miss changes in information processing given simultaneous changes in neurons' baseline firing, response modulation or tuning width. Factors that can modulate baseline firing, such as attention, may have a particularly large influence on population activation. The results have implications for the interpretation of functional imaging signals and for cross-calibration between different methods for measuring neuronal activity.     

A computational model of the development of separate representations of facial identity and expression in the primate visual system

Experimental studies have provided evidence that the visual processing areas of the primate brain represent facial identity and facial expression within different subpopulations of neurons. For example, in non-human primates there is evidence that cells within the inferior temporal gyrus (TE) respond primarily to facial identity, while cells within the superior temporal sulcus (STS) respond to facial expression. More recently, it has been found that the orbitofrontal cortex (OFC) of non-human primates contains some cells that respond exclusively to changes in facial identity, while other cells respond exclusively to facial expression. How might the primate visual system develop physically separate representations of facial identity and expression given that the visual system is always exposed to simultaneous combinations of facial identity and expression during learning? In this paper, a biologically plausible neural network model, VisNet, of the ventral visual pathway is trained on a set of carefully-designed cartoon faces with different identities and expressions. The VisNet model architecture is composed of a hierarchical series of four Self-Organising Maps (SOMs), with associative learning in the feedforward synaptic connections between successive layers. During learning, the network develops separate clusters of cells that respond exclusively to either facial identity or facial expression. We interpret the performance of the network in terms of the learning properties of SOMs, which are able to exploit the statistical indendependence between facial identity and expression.     

Processing of visually presented verbal stimuli in evoked potentials

The ERPs to familiar, unfamiliar, and recently learned words were recorded in 20 right-handed students who have recently studied a course in sensory physiology during performance of two tasks. The first task consisted in processing of different types of words, the second task required recognition of terms from sensory physiology. In the first task, three types of words were presented to a subject: familiar words related and unrelated to a given category and unfamiliar words. Subjects had to press the appropriate button responding to the questions whether they considered the presented word be related to the given category and whether they were sure in their answer. In the second task, terms from recently studied physiological material were presented to the students who had to detect whether those terms were related or unrelated to the given sensory system. Significant differences between late ERP components were revealed by Student test for matched samples. In the first task, component N400 of the ERP to unfamiliar words was higher than that of the ERP to familiar words. It was also higher in ERPs to words unrelated to the given category than in ERPs to related words. However, this component did not differ between ERPs to well and poorly acquired words in the test on sensory physiology. On the contrary, the late positive component was more expressed in ERPs to familiar and better acquired words in both tasks. Brain mechanisms of processing of different types of words are discussed.     

Beauty hinders attention switch in change detection: the role of facial attractiveness and distinctiveness

Background

Recent research has shown that the presence of a task-irrelevant attractive face can induce a transient diversion of attention from a perceptual task that requires covert deployment of attention to one of the two locations. However, it is not known whether this spontaneous appraisal for facial beauty also modulates attention in change detection among multiple locations, where a slower, and more controlled search process is simultaneously affected by the magnitude of a change and the facial distinctiveness. Using the flicker paradigm, this study examines how spontaneous appraisal for facial beauty affects the detection of identity change among multiple faces.

Methodology/Principal Findings

Participants viewed a display consisting of two alternating frames of four faces separated by a blank frame. In half of the trials, one of the faces (target face) changed to a different person. The task of the participant was to indicate whether a change of face identity had occurred. The results showed that (1) observers were less efficient at detecting identity change among multiple attractive faces relative to unattractive faces when the target and distractor faces were not highly distinctive from one another; and (2) it is difficult to detect a change if the new face is similar to the old.

Conclusions/Significance

The findings suggest that attractive faces may interfere with the attention-switch process in change detection. The results also show that attention in change detection was strongly modulated by physical similarity between the alternating faces. Although facial beauty is a powerful stimulus that has well-demonstrated priority, its influence on change detection is easily superseded by low-level image similarity. The visual system appears to take a different approach to facial beauty when a task requires resource-demanding feature comparisons.     

Abnormal FMRI adaptation to unfamiliar faces in a case of developmental prosopamnesia

In rare cases, damage to the temporal lobe causes a selective impairment in the ability to learn new faces, a condition known as prosopamnesia [1]. Here we present the case of an individual with prosopamnesia in the absence of any acquired structural lesion. "C" shows intact processing of simple and complex nonface objects, but her ability to learn new faces is severely impaired. We used a neural marker of perceptual learning known as repetition suppression to examine functioning within C's fusiform face area (FFA), a region of cortex involved in face perception [2]. For comparison, we examined repetition suppression in the scene-selective parahippocampal place area (PPA) [3]. As expected, normal controls showed significant region-specific attenuation of neural activity across repetitions of each stimulus class. C also showed normal attenuation within the PPA to familiar and unfamiliar scenes, and within the FFA to familiar faces. Critically, however, she failed to show any adaptive change within the FFA for repeated unfamiliar faces, despite a face-specific blood-oxygen-dependent response (BOLD) response in her FFA during viewing of face stimuli. Our findings suggest that in developmental prosopamnesia, the FFA cannot maintain stable representations of new faces for subsequent recall or recognition.     

Long-term memory of individual identity in ant queens

Remembering individual identities is part of our own everyday social life. Surprisingly, this ability has recently been shown in two social insects. While paper wasps recognize each other individually through their facial markings, the ant, Pachycondyla villosa, uses chemical cues. In both species, individual recognition is adaptive since it facilitates the maintenance of stable dominance hierarchies among individuals, and thus reduces the cost of conflict within these small societies. Here, we investigated individual recognition in Pachycondyla ants by quantifying the level of aggression between pairs of familiar or unfamiliar queens over time. We show that unrelated founding queens of P. villosa and Pachycondyla inversa store information on the individual identity of other queens and can retrieve it from memory after 24h of separation. Thus, we have documented for the first time that long-term memory of individual identity is present and functional in ants. This novel finding represents an advance in our understanding of the mechanism determining the evolution of cooperation among unrelated individuals.     

Age and the neural network of personal familiarity

Background

Accessing information that defines personally familiar context in real-world situations is essential for the social interactions and the independent functioning of an individual. Personal familiarity is associated with the availability of semantic and episodic information as well as the emotional meaningfulness surrounding a stimulus. These features are known to be associated with neural activity in distinct brain regions across different stimulus conditions (e.g., when perceiving faces, voices, places, objects), which may reflect a shared neural basis. Although perceiving context-rich personal familiarity may appear unchanged in aging on the behavioral level, it has not yet been studied whether this can be supported by neuroimaging data.

Methodology/Principal Findings

We used functional magnetic resonance imaging to investigate the neural network associated with personal familiarity during the perception of personally familiar faces and places. Twelve young and twelve elderly cognitively healthy subjects participated in the study. Both age groups showed a similar activation pattern underlying personal familiarity, predominantly in anterior cingulate and posterior cingulate cortices, irrespective of the stimulus type. The young subjects, but not the elderly subjects demonstrated an additional anterior cingulate deactivation when perceiving unfamiliar stimuli.

Conclusions/Significance

Although we found evidence for an age-dependent reduction in frontal cortical deactivation, our data show that there is a stimulus-independent neural network associated with personal familiarity of faces and places, which is less susceptible to aging-related changes.