The face is not an empty canvas: how facial expressions interact with facial appearance

Faces are not simply blank canvases upon which facial expressions write their emotional messages. In fact, facial appearance and facial movement are both important social signalling systems in their own right. We here provide multiple lines of evidence for the notion that the social signals derived from facial appearance on the one hand and facial movement on the other interact in a complex manner, sometimes reinforcing and sometimes contradicting one another. Faces provide information on who a person is. Sex, age, ethnicity, personality and other characteristics that can define a person and the social group the person belongs to can all be derived from the face alone. The present article argues that faces interact with the perception of emotion expressions because this information informs a decoder''s expectations regarding an expresser''s probable emotional reactions. Facial appearance also interacts more directly with the interpretation of facial movement because some of the features that are used to derive personality or sex information are also features that closely resemble certain emotional expressions, thereby enhancing or diluting the perceived strength of particular expressions.     

Emotional facial expression detection in the peripheral visual field

Background

In everyday life, signals of danger, such as aversive facial expressions, usually appear in the peripheral visual field. Although facial expression processing in central vision has been extensively studied, this processing in peripheral vision has been poorly studied.

Methodology/Principal Findings

Using behavioral measures, we explored the human ability to detect fear and disgust vs. neutral expressions and compared it to the ability to discriminate between genders at eccentricities up to 40°. Responses were faster for the detection of emotion compared to gender. Emotion was detected from fearful faces up to 40° of eccentricity.

Conclusions

Our results demonstrate the human ability to detect facial expressions presented in the far periphery up to 40° of eccentricity. The increasing advantage of emotion compared to gender processing with increasing eccentricity might reflect a major implication of the magnocellular visual pathway in facial expression processing. This advantage may suggest that emotion detection, relative to gender identification, is less impacted by visual acuity and within-face crowding in the periphery. These results are consistent with specific and automatic processing of danger-related information, which may drive attention to those messages and allow for a fast behavioral reaction.     

Neuroticism Delays Detection of Facial Expressions

The rapid detection of emotional signals from facial expressions is fundamental for human social interaction. The personality factor of neuroticism modulates the processing of various types of emotional facial expressions; however, its effect on the detection of emotional facial expressions remains unclear. In this study, participants with high- and low-neuroticism scores performed a visual search task to detect normal expressions of anger and happiness, and their anti-expressions within a crowd of neutral expressions. Anti-expressions contained an amount of visual changes equivalent to those found in normal expressions compared to neutral expressions, but they were usually recognized as neutral expressions. Subjective emotional ratings in response to each facial expression stimulus were also obtained. Participants with high-neuroticism showed an overall delay in the detection of target facial expressions compared to participants with low-neuroticism. Additionally, the high-neuroticism group showed higher levels of arousal to facial expressions compared to the low-neuroticism group. These data suggest that neuroticism modulates the detection of emotional facial expressions in healthy participants; high levels of neuroticism delay overall detection of facial expressions and enhance emotional arousal in response to facial expressions.     

Immune responses of zebrafish (Danio rerio) induced by bath-vaccination with a live attenuated Vibrio anguillarum vaccine candidate

A fish vaccine candidate, live attenuated Vibrio anguillarum, which can protect fish from vibriosis, was established in our laboratory. In this study, the protective immunological mechanism of live attenuated V. anguillarum was investigated in zebrafish as a model animal. After bath-vaccinated with the live attenuated strain, zebrafish were challenged with wild pathogenic strain to test the immunoprotection of the live attenuated strain. As the results, specific antibody response of fish against V. anguillarum was found to gradually increase during 28 days post-vaccination, and remarkable protection was showed with a high relative protection survival (RPS) of about 90%. Moreover, the vaccination changed the expressions of several immune-related genes in the spleens and livers of zebrafish. Among them, the expressions of pro-inflammatory factors such as IL-1 and IL-8 were tenderly up-regulated with about 3-4 fold in 1-7 days post-vaccination, while MHC II rose to a peak level of 4-fold in 7th day post-vaccination. These results gave some important messages about the mechanism of specific protection induced by live attenuated V. anguillarum and showed the availability of zebrafish model in the evaluation of the vaccine candidate.     

Social communication in siamangs (<Emphasis Type="Italic">Symphalangus syndactylus</Emphasis>): use of gestures and facial expressions

The current study represents the first systematic investigation of the social communication of captive siamangs (Symphalangus syndactylus). The focus was on intentional signals, including tactile and visual gestures, as well as facial expressions and actions. Fourteen individuals from different groups were observed and the signals used by individuals were recorded. Thirty-one different signals, consisting of 12 tactile gestures, 8 visual gestures, 7 actions, and 4 facial expressions, were observed, with tactile gestures and facial expressions appearing most frequently. The range of the signal repertoire increased steadily until the age of six, but declined afterwards in adults. The proportions of the different signal categories used within communicative interactions, in particular actions and facial expressions, also varied depending on age. Group differences could be traced back mainly to social factors or housing conditions. Differences in the repertoire of males and females were most obvious in the sexual context. Overall, most signals were used flexibly, with the majority performed in three or more social contexts and almost one-third of signals used in combination with other signals. Siamangs also adjusted their signals appropriately for the recipient, for example, using visual signals most often when the recipient was already attending (audience effects). These observations are discussed in the context of siamang ecology, social structure, and cognition.To see video sequences of signals described here, please go to     

The MPI facial expression database--a validated database of emotional and conversational facial expressions

The ability to communicate is one of the core aspects of human life. For this, we use not only verbal but also nonverbal signals of remarkable complexity. Among the latter, facial expressions belong to the most important information channels. Despite the large variety of facial expressions we use in daily life, research on facial expressions has so far mostly focused on the emotional aspect. Consequently, most databases of facial expressions available to the research community also include only emotional expressions, neglecting the largely unexplored aspect of conversational expressions. To fill this gap, we present the MPI facial expression database, which contains a large variety of natural emotional and conversational expressions. The database contains 55 different facial expressions performed by 19 German participants. Expressions were elicited with the help of a method-acting protocol, which guarantees both well-defined and natural facial expressions. The method-acting protocol was based on every-day scenarios, which are used to define the necessary context information for each expression. All facial expressions are available in three repetitions, in two intensities, as well as from three different camera angles. A detailed frame annotation is provided, from which a dynamic and a static version of the database have been created. In addition to describing the database in detail, we also present the results of an experiment with two conditions that serve to validate the context scenarios as well as the naturalness and recognizability of the video sequences. Our results provide clear evidence that conversational expressions can be recognized surprisingly well from visual information alone. The MPI facial expression database will enable researchers from different research fields (including the perceptual and cognitive sciences, but also affective computing, as well as computer vision) to investigate the processing of a wider range of natural facial expressions.     

When Age Matters: Differences in Facial Mimicry and Autonomic Responses to Peers' Emotions in Teenagers and Adults

Age-group membership effects on explicit emotional facial expressions recognition have been widely demonstrated. In this study we investigated whether Age-group membership could also affect implicit physiological responses, as facial mimicry and autonomic regulation, to observation of emotional facial expressions. To this aim, facial Electromyography (EMG) and Respiratory Sinus Arrhythmia (RSA) were recorded from teenager and adult participants during the observation of facial expressions performed by teenager and adult models. Results highlighted that teenagers exhibited greater facial EMG responses to peers'' facial expressions, whereas adults showed higher RSA-responses to adult facial expressions. The different physiological modalities through which young and adults respond to peers'' emotional expressions are likely to reflect two different ways to engage in social interactions with coetaneous. Findings confirmed that age is an important and powerful social feature that modulates interpersonal interactions by influencing low-level physiological responses.     

Correlated evolution of brain regions involved in producing and processing facial expressions in anthropoid primates

Anthropoid primates are distinguished from other mammals by having relatively large primary visual cortices (V1) and complex facial expressions. We present a comparative test of the hypothesis that facial expression processing coevolved with the expansion of V1 in anthropoids. Previously published data were analysed using phylogenetic comparative methods. The results of our study suggest a pattern of correlated evolution linking social group size, facial motor control and cortical visual processing in catarrhines, but not platyrrhines. Catarrhines that live in relatively large social groups tended to have relatively large facial motor nuclei, and relatively large primary visual cortices. We conclude that catarrhine brains are adapted for producing and processing complex facial displays.     

Individual Differences in the Recognition of Facial Expressions: An Event-Related Potentials Study

Previous studies have shown that early posterior components of event-related potentials (ERPs) are modulated by facial expressions. The goal of the current study was to investigate individual differences in the recognition of facial expressions by examining the relationship between ERP components and the discrimination of facial expressions. Pictures of 3 facial expressions (angry, happy, and neutral) were presented to 36 young adults during ERP recording. Participants were asked to respond with a button press as soon as they recognized the expression depicted. A multiple regression analysis, where ERP components were set as predictor variables, assessed hits and reaction times in response to the facial expressions as dependent variables. The N170 amplitudes significantly predicted for accuracy of angry and happy expressions, and the N170 latencies were predictive for accuracy of neutral expressions. The P2 amplitudes significantly predicted reaction time. The P2 latencies significantly predicted reaction times only for neutral faces. These results suggest that individual differences in the recognition of facial expressions emerge from early components in visual processing.     

Dogs Evaluate Threatening Facial Expressions by Their Biological Validity – Evidence from Gazing Patterns

Appropriate response to companions’ emotional signals is important for all social creatures. The emotional expressions of humans and non-human animals have analogies in their form and function, suggesting shared evolutionary roots, but very little is known about how animals other than primates view and process facial expressions. In primates, threat-related facial expressions evoke exceptional viewing patterns compared with neutral or positive stimuli. Here, we explore if domestic dogs (Canis familiaris) have such an attentional bias toward threatening social stimuli and whether observed emotional expressions affect dogs’ gaze fixation distribution among the facial features (eyes, midface and mouth). We recorded the voluntary eye gaze of 31 domestic dogs during viewing of facial photographs of humans and dogs with three emotional expressions (threatening, pleasant and neutral). We found that dogs’ gaze fixations spread systematically among facial features. The distribution of fixations was altered by the seen expression, but eyes were the most probable targets of the first fixations and gathered longer looking durations than mouth regardless of the viewed expression. The examination of the inner facial features as a whole revealed more pronounced scanning differences among expressions. This suggests that dogs do not base their perception of facial expressions on the viewing of single structures, but the interpretation of the composition formed by eyes, midface and mouth. Dogs evaluated social threat rapidly and this evaluation led to attentional bias, which was dependent on the depicted species: threatening conspecifics’ faces evoked heightened attention but threatening human faces instead an avoidance response. We propose that threatening signals carrying differential biological validity are processed via distinctive neurocognitive pathways. Both of these mechanisms may have an adaptive significance for domestic dogs. The findings provide a novel perspective on understanding the processing of emotional expressions and sensitivity to social threat in non-primates.     

Role of facial expressions in social interactions

The expressions we see in the faces of others engage a number of different cognitive processes. Emotional expressions elicit rapid responses, which often imitate the emotion in the observed face. These effects can even occur for faces presented in such a way that the observer is not aware of them. We are also very good at explicitly recognizing and describing the emotion being expressed. A recent study, contrasting human and humanoid robot facial expressions, suggests that people can recognize the expressions made by the robot explicitly, but may not show the automatic, implicit response. The emotional expressions presented by faces are not simply reflexive, but also have a communicative component. For example, empathic expressions of pain are not simply a reflexive response to the sight of pain in another, since they are exaggerated when the empathizer knows he or she is being observed. It seems that we want people to know that we are empathic. Of especial importance among facial expressions are ostensive gestures such as the eyebrow flash, which indicate the intention to communicate. These gestures indicate, first, that the sender is to be trusted and, second, that any following signals are of importance to the receiver.     

Facial expressions as honest signals of cooperative intent in a one-shot anonymous Prisoner's Dilemma game

Previous research has posited that facial expressions of emotion may serve as honest signals of cooperation. Although findings from several empirical studies support this position, prior studies have not used comprehensive and dynamic measures of facial expression as potential predictors of behaviorally defined cooperation. The authors investigated (a) specific positive and negative facial actions displayed among strangers immediately following verbal promises of commitment within an unrestricted acquaintance period and (b) anonymous, behaviorally defined decisions of cooperation or defection in a one-shot, two-person Prisoner's Dilemma game occurring directly following the acquaintance period. The Facial Action Coding System [Ekman P. & Friesen W.V. (1978). Facial Action Coding System. Palo Alto, CA: Consulting Psychology Press] was used to measure affect-related facial actions. It was found that facial actions related to enjoyment were predictive of cooperative decisions within dyads; additionally, facial actions related to contempt were predictive of noncooperative decisions within dyads. Furthermore, and consistent with previous works, participants were able to accurately predict their partner's decisions following the acquaintance period. These results suggest that facial actions may function as honest signals of cooperative intent. These findings also provide a possible explanation for the association between subjective affective experience and facial expression that advances understanding of cooperative behavior.     

Involuntary facial expression processing: extracting information from two simultaneously presented faces

Facial expressions play an important role in successful social interactions, with previous research suggesting that facial expressions may be processed involuntarily. In the current study, we investigate whether involuntary processing of facial expressions would also occur when facial expression distractors are simultaneously presented in the same spatial location as facial expression targets. Targets and distractors from another stimulus class (lions) were also used. Results indicated that angry facial expression distractors interfered more than neutral face distractors with the ability to respond to both face and lion targets. These findings suggest that information from angry facial expressions can be extracted rapidly from a very brief presentation (50 ms), providing compelling evidence that angry facial expressions are processed involuntarily.     

Social Use of Facial Expressions in Hylobatids

Non-human primates use various communicative means in interactions with others. While primate gestures are commonly considered to be intentionally and flexibly used signals, facial expressions are often referred to as inflexible, automatic expressions of affective internal states. To explore whether and how non-human primates use facial expressions in specific communicative interactions, we studied five species of small apes (gibbons) by employing a newly established Facial Action Coding System for hylobatid species (GibbonFACS). We found that, despite individuals often being in close proximity to each other, in social (as opposed to non-social contexts) the duration of facial expressions was significantly longer when gibbons were facing another individual compared to non-facing situations. Social contexts included grooming, agonistic interactions and play, whereas non-social contexts included resting and self-grooming. Additionally, gibbons used facial expressions while facing another individual more often in social contexts than non-social contexts where facial expressions were produced regardless of the attentional state of the partner. Also, facial expressions were more likely ‘responded to’ by the partner’s facial expressions when facing another individual than non-facing. Taken together, our results indicate that gibbons use their facial expressions differentially depending on the social context and are able to use them in a directed way in communicative interactions with other conspecifics.     

Testosterone reactivity to facial display of emotions in men and women

Previous studies have examined testosterone's role in regulating the processing of facial displays of emotions (FDEs). However, the reciprocal process – the influence of FDEs, an evolutionarily ancient and potent class of social signals, on the secretion of testosterone – has not yet been studied. To address this gap, we examined the effects of emotional content and sex of facial stimuli in modulating endogenous testosterone fluctuations, as well as sex differences in the endocrine responses to faces. One hundred and sixty-four young healthy men and women were exposed, in a between-subjects design, to happy or angry same-sex or opposite-sex facial expressions. Results showed that in both men (n = 85) and women (n = 79), extended exposure to faces of the opposite sex, regardless of their apparent emotional content, was accompanied by an accumulation in salivary testosterone when compared to exposure to faces of the same sex. Furthermore, testosterone change in women exposed to angry expressions was greater than testosterone change in women exposed to happy expressions. These results add emotional facial stimuli to the collection of social signals that modulate endocrine status, and are discussed with regard to the evolutionary roles of testosterone.     

When Early Experiences Build a Wall to Others’ Emotions: An Electrophysiological and Autonomic Study

Facial expression of emotions is a powerful vehicle for communicating information about others’ emotional states and it normally induces facial mimicry in the observers. The aim of this study was to investigate if early aversive experiences could interfere with emotion recognition, facial mimicry, and with the autonomic regulation of social behaviors. We conducted a facial emotion recognition task in a group of “street-boys” and in an age-matched control group. We recorded facial electromyography (EMG), a marker of facial mimicry, and respiratory sinus arrhythmia (RSA), an index of the recruitment of autonomic system promoting social behaviors and predisposition, in response to the observation of facial expressions of emotions. Results showed an over-attribution of anger, and reduced EMG responses during the observation of both positive and negative expressions only among street-boys. Street-boys also showed lower RSA after observation of facial expressions and ineffective RSA suppression during presentation of non-threatening expressions. Our findings suggest that early aversive experiences alter not only emotion recognition but also facial mimicry of emotions. These deficits affect the autonomic regulation of social behaviors inducing lower social predisposition after the visualization of facial expressions and an ineffective recruitment of defensive behavior in response to non-threatening expressions.     

Optimal Geometrical Set for Automated Marker Placement to Virtualized Real-Time Facial Emotions

In recent years, real-time face recognition has been a major topic of interest in developing intelligent human-machine interaction systems. Over the past several decades, researchers have proposed different algorithms for facial expression recognition, but there has been little focus on detection in real-time scenarios. The present work proposes a new algorithmic method of automated marker placement used to classify six facial expressions: happiness, sadness, anger, fear, disgust, and surprise. Emotional facial expressions were captured using a webcam, while the proposed algorithm placed a set of eight virtual markers on each subject’s face. Facial feature extraction methods, including marker distance (distance between each marker to the center of the face) and change in marker distance (change in distance between the original and new marker positions), were used to extract three statistical features (mean, variance, and root mean square) from the real-time video sequence. The initial position of each marker was subjected to the optical flow algorithm for marker tracking with each emotional facial expression. Finally, the extracted statistical features were mapped into corresponding emotional facial expressions using two simple non-linear classifiers, K-nearest neighbor and probabilistic neural network. The results indicate that the proposed automated marker placement algorithm effectively placed eight virtual markers on each subject’s face and gave a maximum mean emotion classification rate of 96.94% using the probabilistic neural network.     

Emotional Modulation of the Attentional Blink Is Awareness-Dependent

It is well known that emotion can modulate attentional processes. Previous studies have shown that even under restricted awareness, emotional facial expressions (especially threat-related) can guide the direction of spatial attention. However, it remains unclear whether emotional facial expressions under restricted awareness can affect temporal attention. To address this issue, we used a modified attentional blink (AB) paradigm in which masked (Experiment 1) or unmasked (Experiment 2) emotional faces (fearful or neutral) were presented before the AB sequence. We found that, in comparison with neutral faces, masked fearful faces significantly decreased the AB magnitude (Experiment 1), whereas unmasked fearful faces significantly increased the AB magnitude (Experiment 2). These results indicate that effects of emotional expression on the AB are modulated by the level of awareness.     

Congenital facial palsy and emotion processing: The case of Moebius syndrome

According to the Darwinian perspective, facial expressions of emotions evolved to quickly communicate emotional states and would serve adaptive functions that promote social interactions. Embodied cognition theories suggest that we understand others' emotions by reproducing the perceived expression in our own facial musculature (facial mimicry) and the mere observation of a facial expression can evoke the corresponding emotion in the perceivers. Consequently, the inability to form facial expressions would affect the experience of emotional understanding. In this review, we aimed at providing account on the link between the lack of emotion production and the mechanisms of emotion processing. We address this issue by taking into account Moebius syndrome, a rare neurological disorder that primarily affects the muscles controlling facial expressions. Individuals with Moebius syndrome are born with facial paralysis and inability to form facial expressions. This makes them the ideal population to study whether facial mimicry is necessary for emotion understanding. Here, we discuss behavioral ambiguous/mixed results on emotion recognition deficits in Moebius syndrome suggesting the need to investigate further aspects of emotional processing such as the physiological responses associated with the emotional experience during developmental age.     

Long‐distance signals influence assessment of close range mating displays in the field cricket,Gryllus integer

Male sexual displays often include components detected across long distances, and those perceived only at close range. Understanding what information females gain from each component of a complex display and how they use these signals to make decisions are questions of major interest in sexual selection research. We evaluated content‐based hypotheses (‘redundant signals’ and ‘multiple messages’) for the courtship displays of field crickets (Gryllus integer) by measuring female responses to males' long‐distance calling song (calls) and close‐range chemical cues. Females' responses to a male's calls and chemical cues were uncorrelated, supporting the ‘multiple messages’ hypothesis. We also tested the ‘inter‐signal interaction’ hypothesis by investigating how long‐distance calls influence evaluation of close‐range courtship. The relationship between long‐ and close‐range signals was complex and conditional: females accepted close‐range courtship more quickly after exposure to attractive calling song than they did after exposure to either unattractive calling song or silence, and unattractive calls were no more or less effective than silence. This inter‐signal interaction could affect our understanding of mate choice in species with multiple mating signals because it implies that females may save time and energy by not assessing the close‐range signals of attractive long‐distance signalers. © 2010 The Linnean Society of London, Biological Journal of the Linnean Society, 2010, 100 , 856–865.