Young infants' neural processing of objects is affected by eye gaze direction and emotional expression

Eye gaze is an important social cue which is used to determine another person's focus of attention and intention to communicate. In combination with a fearful facial expression eye gaze can also signal threat in the environment. The ability to detect and understand others' social signals is essential in order to avoid danger and enable social evaluation. It has been a matter of debate when infants are able to use gaze cues and emotional facial expressions in reference to external objects. Here we demonstrate that by 3 months of age the infant brain differentially responds to objects as a function of how other people are reacting to them. Using event-related electrical brain potentials (ERPs), we show that an indicator of infants' attention is enhanced by an adult's expression of fear toward an unfamiliar object. The infant brain showed an increased Negative central (Nc) component toward objects that had been previously cued by an adult's eye gaze and frightened facial expression. Our results further suggest that infants' sensitivity cannot be due to a general arousal elicited by a frightened face with eye gaze directed at an object. The neural attention system of 3 month old infants is sensitive to an adult's eye gaze direction in combination with a fearful expression. This early capacity may lay the foundation for the development of more sophisticated social skills such as social referencing, language, and theory of mind.     

Attention to Eye Contact in the West and East: Autonomic Responses and Evaluative Ratings

Eye contact has a fundamental role in human social interaction. The special appearance of the human eye (i.e., white sclera contrasted with a coloured iris) implies the importance of detecting another person''s face through eye contact. Empirical studies have demonstrated that faces making eye contact are detected quickly and processed preferentially (i.e., the eye contact effect). Such sensitivity to eye contact seems to be innate and universal among humans; however, several studies suggest that cultural norms affect eye contact behaviours. For example, Japanese individuals exhibit less eye contact than do individuals from Western European or North American cultures. However, how culture modulates eye contact behaviour is unclear. The present study investigated cultural differences in autonomic correlates of attentional orienting (i.e., heart rate) and looking time. Additionally, we examined evaluative ratings of eye contact with another real person, displaying an emotionally neutral expression, between participants from Western European (Finnish) and East Asian (Japanese) cultures. Our results showed that eye contact elicited stronger heart rate deceleration responses (i.e., attentional orienting), shorter looking times, and higher ratings of subjective feelings of arousal as compared to averted gaze in both cultures. Instead, cultural differences in the eye contact effect were observed in various evaluative responses regarding the stimulus faces (e.g., facial emotion, approachability etc.). The rating results suggest that individuals from an East Asian culture perceive another''s face as being angrier, unapproachable, and unpleasant when making eye contact as compared to individuals from a Western European culture. The rating results also revealed that gaze direction (direct vs. averted) could influence perceptions about another person''s facial affect and disposition. These results suggest that cultural differences in eye contact behaviour emerge from differential display rules and cultural norms, as opposed to culture affecting eye contact behaviour directly at the physiological level.     

Salience of emotional displays of danger and contagion in faces is enhanced when progesterone levels are raised

Findings from previous studies of hormone-mediated behavior in women suggest that raised progesterone level increases the probability of behaviors that will reduce the likelihood of disruption to fetal development during pregnancy (e.g. increased avoidance of sources of contagion). Here, we tested women's (N=52) sensitivity to potential cues to nearby sources of contagion (disgusted facial expressions with averted gaze) and nearby physical threat (fearful facial expressions with averted gaze) at two points in the menstrual cycle differing in progesterone level. Women demonstrated a greater tendency to perceive fearful and disgusted expressions with averted gaze as more intense than those with direct gaze when their progesterone level was relatively high. By contrast, change in progesterone level was not associated with any change in perceptions of happy expressions with direct and averted gaze, indicating that our findings for disgusted and fearful expressions were not due to a general response bias. Collectively, our findings suggest women are more sensitive to facial cues signalling nearby contagion and physical threat when raised progesterone level prepares the body for pregnancy.     

Watch Out! Magnetoencephalographic Evidence for Early Modulation of Attention Orienting by Fearful Gaze Cueing

Others’ gaze and emotional facial expression are important cues for the process of attention orienting. Here, we investigated with magnetoencephalography (MEG) whether the combination of averted gaze and fearful expression may elicit a selectively early effect of attention orienting on the brain responses to targets. We used the direction of gaze of centrally presented fearful and happy faces as the spatial attention orienting cue in a Posner-like paradigm where the subjects had to detect a target checkerboard presented at gazed-at (valid trials) or non gazed-at (invalid trials) locations of the screen. We showed that the combination of averted gaze and fearful expression resulted in a very early attention orienting effect in the form of additional parietal activity between 55 and 70 ms for the valid versus invalid targets following fearful gaze cues. No such effect was obtained for the targets following happy gaze cues. This early cue-target validity effect selective of fearful gaze cues involved the left superior parietal region and the left lateral middle occipital region. These findings provide the first evidence for an effect of attention orienting induced by fearful gaze in the time range of C1. In doing so, they demonstrate the selective impact of combined gaze and fearful expression cues in the process of attention orienting.     

Cortical arousal and social intimacy in the human female under different conditions of eye contact

The EEG was monitored from pairs of female subjects while they engaged in varied eye contact under experimenter instruction (direct gaze, smile, averted gaze). The nine conditions of gaze were related monotonically to EEG abundance (9.5–20.0 Hz). It is suggested that the gradient of arousal or activation so obtained is evidence of a physiological substrate of social intimacy.     

Cultural Modulation of Face and Gaze Scanning in Young Children

Previous research has demonstrated that the way human adults look at others’ faces is modulated by their cultural background, but very little is known about how such a culture-specific pattern of face gaze develops. The current study investigated the role of cultural background on the development of face scanning in young children between the ages of 1 and 7 years, and its modulation by the eye gaze direction of the face. British and Japanese participants’ eye movements were recorded while they observed faces moving their eyes towards or away from the participants. British children fixated more on the mouth whereas Japanese children fixated more on the eyes, replicating the results with adult participants. No cultural differences were observed in the differential responses to direct and averted gaze. The results suggest that different patterns of face scanning exist between different cultures from the first years of life, but differential scanning of direct and averted gaze associated with different cultural norms develop later in life.     

Searching for asymmetries in the detection of gaze contact versus averted gaze under different head views: a behavioural study

Eye contact is a crucial social cue constituting a frequent preliminary to interaction. Thus, the perception of others' gaze may be associated with specific processes beginning with asymmetries in the detection of direct versus averted gaze. We tested this hypothesis in two behavioural experiments using realistic eye stimuli in a visual search task. We manipulated the head orientation (frontal or deviated) and the visual field (right or left) in which the target appeared at display onset. We found that direct gaze targets presented among averted gaze distractors were detected faster and better than averted gaze targets among direct gaze distractors, but only when the head was deviated. Moreover, direct gaze targets were detected very quickly and efficiently regardless of head orientation and visual field, whereas the detection of averted gaze was strongly modulated by these factors. These results suggest that gaze contact has precedence over contextual information such as head orientation and visual field.     

Cues for early social skills: direct gaze modulates newborns' recognition of talking faces

Previous studies showed that, from birth, speech and eye gaze are two important cues in guiding early face processing and social cognition. These studies tested the role of each cue independently; however, infants normally perceive speech and eye gaze together. Using a familiarization-test procedure, we first familiarized newborn infants (n = 24) with videos of unfamiliar talking faces with either direct gaze or averted gaze. Newborns were then tested with photographs of the previously seen face and of a new one. The newborns looked longer at the face that previously talked to them, but only in the direct gaze condition. These results highlight the importance of both speech and eye gaze as socio-communicative cues by which infants identify others. They suggest that gaze and infant-directed speech, experienced together, are powerful cues for the development of early social skills.     

Neural Signals Evoked by Stimuli of Increasing Social Scene Complexity Are Detectable at the Single-Trial Level and Right Lateralized

Classification of neural signals at the single-trial level and the study of their relevance in affective and cognitive neuroscience are still in their infancy. Here we investigated the neurophysiological correlates of conditions of increasing social scene complexity using 3D human models as targets of attention, which may also be important in autism research. Challenging single-trial statistical classification of EEG neural signals was attempted for detection of oddball stimuli with increasing social scene complexity. Stimuli had an oddball structure and were as follows: 1) flashed schematic eyes, 2) simple 3D faces flashed between averted and non-averted gaze (only eye position changing), 3) simple 3D faces flashed between averted and non-averted gaze (head and eye position changing), 4) animated avatar alternated its gaze direction to the left and to the right (head and eye position), 5) environment with 4 animated avatars all of which change gaze and one of which is the target of attention. We found a late (> 300 ms) neurophysiological oddball correlate for all conditions irrespective of their complexity as assessed by repeated measures ANOVA. We attempted single-trial detection of this signal with automatic classifiers and obtained a significant balanced accuracy classification of around 79%, which is noteworthy given the amount of scene complexity. Lateralization analysis showed a specific right lateralization only for more complex realistic social scenes. In sum, complex ecological animations with social content elicit neurophysiological events which can be characterized even at the single-trial level. These signals are right lateralized. These finding paves the way for neuroscientific studies in affective neuroscience based on complex social scenes, and given the detectability at the single trial level this suggests the feasibility of brain computer interfaces that can be applied to social cognition disorders such as autism.     

Facial-expression and gaze-selective responses in the monkey amygdala

The social behavior of both human and nonhuman primates relies on specializations for the recognition of individuals, their facial expressions, and their direction of gaze. A broad network of cortical and subcortical structures has been implicated in face processing, yet it is unclear whether co-occurring dimensions of face stimuli, such as expression and direction of gaze, are processed jointly or independently by anatomically and functionally segregated neural structures. Awake macaques were presented with a set of monkey faces displaying aggressive, neutral, and appeasing expressions with head and eyes either averted or directed. BOLD responses to these faces as compared to Fourier-phase-scrambled images revealed widespread activation of the superior temporal sulcus and inferotemporal cortex and included activity in the amygdala. The different dimensions of the face stimuli elicited distinct activation patterns among the amygdaloid nuclei. The basolateral amygdala, including the lateral, basal, and accessory basal nuclei, produced a stronger response for threatening than appeasing expressions. The central nucleus and bed nucleus of the stria terminalis responded more to averted than directed-gaze faces. Independent behavioral measures confirmed that faces with averted gaze were more arousing, suggesting the activity in the central nucleus may be related to attention and arousal.     

Visual Search for Human Gaze Direction by a Chimpanzee (Pan troglodytes)

Background

Humans detect faces with direct gazes among those with averted gazes more efficiently than they detect faces with averted gazes among those with direct gazes. We examined whether this “stare-in-the-crowd” effect occurs in chimpanzees (Pan troglodytes), whose eye morphology differs from that of humans (i.e., low-contrast eyes, dark sclera).

Methodology/Principal Findings

An adult female chimpanzee was trained to search for an odd-item target (front view of a human face) among distractors that differed from the target only with respect to the direction of the eye gaze. During visual-search testing, she performed more efficiently when the target was a direct-gaze face than when it was an averted-gaze face. This direct-gaze superiority was maintained when the faces were inverted and when parts of the face were scrambled. Subsequent tests revealed that gaze perception in the chimpanzee was controlled by the contrast between iris and sclera, as in humans, but that the chimpanzee attended only to the position of the iris in the eye, irrespective of head direction.

Conclusion/Significance

These results suggest that the chimpanzee can discriminate among human gaze directions and are more sensitive to direct gazes. However, limitations in the perception of human gaze by the chimpanzee are suggested by her inability to completely transfer her performance to faces showing a three-quarter view.     

Social status gates social attention in humans

Humans tend to shift attention in response to the averted gaze of a face they are fixating, a phenomenon known as gaze cuing. In the present paper, we aimed to address whether the social status of the cuing face modulates this phenomenon. Participants were asked to look at the faces of 16 individuals and read fictive curriculum vitae associated with each of them that could describe the person as having a high or low social status. The association between each specific face and either high or low social status was counterbalanced between participants. The same faces were then used as stimuli in a gaze-cuing task. The results showed a greater gaze-cuing effect for high-status faces than for low-status faces, independently of the specific identity of the face. These findings confirm previous evidence regarding the important role of social factors in shaping social attention and show that a modulation of gaze cuing can be observed even when knowledge about social status is acquired through episodic learning.     

Racial group membership is associated to gaze-mediated orienting in Italy

Viewing a face with averted gaze results in a spatial shift of attention in the corresponding direction, a phenomenon defined as gaze-mediated orienting. In the present paper, we investigated whether this effect is influenced by social factors. Across three experiments, White and Black participants were presented with faces of White and Black individuals. A modified spatial cueing paradigm was used in which a peripheral target stimulus requiring a discrimination response was preceded by a noninformative gaze cue. Results showed that Black participants shifted attention to the averted gaze of both ingroup and outgroup faces, whereas White participants selectively shifted attention only in response to individuals of their same group. Interestingly, the modulatory effect of social factors was context-dependent and emerged only when group membership was situationally salient to participants. It was hypothesized that differences in the relative social status of the two groups might account for the observed asymmetry between White and Black participants. A final experiment ruled out an alternative explanation based on differences in perceptual familiarity with the face stimuli. Overall, these findings strengthen the idea that gaze-mediated orienting is a socially-connoted phenomenon.     

Individual Differences in Personality Predict How People Look at Faces

Background

Determining the ways in which personality traits interact with contextual determinants to shape social behavior remains an important area of empirical investigation. The specific personality trait of neuroticism has been related to characteristic negative emotionality and associated with heightened attention to negative, emotionally arousing environmental signals. However, the mechanisms by which this personality trait may shape social behavior remain largely unspecified.

Methodology/Principal Findings

We employed eye tracking to investigate the relationship between characteristics of visual scanpaths in response to emotional facial expressions and individual differences in personality. We discovered that the amount of time spent looking at the eyes of fearful faces was positively related to neuroticism.

Conclusions/Significance

This finding is discussed in relation to previous behavioral research relating personality to selective attention for trait-congruent emotional information, neuroimaging studies relating differences in personality to amygdala reactivity to socially relevant stimuli, and genetic studies suggesting linkages between the serotonin transporter gene and neuroticism. We conclude that personality may be related to interpersonal interaction by shaping aspects of social cognition as basic as eye contact. In this way, eye gaze represents a possible behavioral link in a complex relationship between genes, brain function, and personality.     

Early environment shapes the development of gaze aversion by wild bonnet macaques (<Emphasis Type="Italic">Macaca radiata</Emphasis>)

Among many species of primates, staring is perceived as a sign of aggression and averting the gaze usually serves to reduce such conflict. The current study conducted in southern India documented developmental differences among wild bonnet macaques (Macaca radiata) in their latency to gaze avert after establishing eye contact with other individuals. Feeding stations were used to gather macaques within a small area to facilitate the video recording of group dynamics and eye contact between subordinate and dominant individuals. Individuals were grouped into three age classes: juveniles, subadult males, and adult males. Comparisons were also made between urban and forest dwelling troops. In the forest, juveniles established eye contact with older males for significantly longer periods of time than did adults. A linear trend was observed in which the latency to gaze avert after establishing eye contact decreased with age. This trend was not evident in the urban troops, for which the latency to gaze avert did not change significantly with age. Urban juveniles were also more likely to be chased when they established eye contact with adults compared with their forest counterparts. These differences could be the result of increased predatory risk in the forest setting — the necessity for heightened predator vigilance in forests may reduce the frequency with which juveniles are monitored and chased or attacked as a result of their eye contact. Conversely, the rarity of predators in the city may engender more intense aggressive behavior between monkeys, accelerating the rate of learning to signal appeasement to dominant males.     

Speaking and Listening with the Eyes: Gaze Signaling during Dyadic Interactions

Cognitive scientists have long been interested in the role that eye gaze plays in social interactions. Previous research suggests that gaze acts as a signaling mechanism and can be used to control turn-taking behaviour. However, early research on this topic employed methods of analysis that aggregated gaze information across an entire trial (or trials), which masks any temporal dynamics that may exist in social interactions. More recently, attempts have been made to understand the temporal characteristics of social gaze but little research has been conducted in a natural setting with two interacting participants. The present study combines a temporally sensitive analysis technique with modern eye tracking technology to 1) validate the overall results from earlier aggregated analyses and 2) provide insight into the specific moment-to-moment temporal characteristics of turn-taking behaviour in a natural setting. Dyads played two social guessing games (20 Questions and Heads Up) while their eyes were tracked. Our general results are in line with past aggregated data, and using cross-correlational analysis on the specific gaze and speech signals of both participants we found that 1) speakers end their turn with direct gaze at the listener and 2) the listener in turn begins to speak with averted gaze. Convergent with theoretical models of social interaction, our data suggest that eye gaze can be used to signal both the end and the beginning of a speaking turn during a social interaction. The present study offers insight into the temporal dynamics of live dyadic interactions and also provides a new method of analysis for eye gaze data when temporal relationships are of interest.     

Atypical gaze patterns in children and adults with autism spectrum disorders dissociated from developmental changes in gaze behaviour

Eye tracking has been used to investigate gaze behaviours in individuals with autism spectrum disorder (ASD). However, traditional analysis has yet to find behavioural characteristics shared by both children and adults with ASD. To distinguish core ASD gaze behaviours from those that change with development, we examined temporo-spatial gaze patterns in children and adults with and without ASD while they viewed video clips. We summarized the gaze patterns of 104 participants using multidimensional scaling so that participants with similar gaze patterns would cluster together in a two-dimensional plane. Control participants clustered in the centre, reflecting a standard gaze behaviour, whereas participants with ASD were distributed around the periphery. Moreover, children and adults were separated on the plane, thereby showing a clear effect of development on gaze behaviours. Post hoc frame-by-frame analyses revealed the following findings: (i) both ASD groups shifted their gaze away from a speaker earlier than the control groups; (ii) both ASD groups showed a particular preference for letters; and (iii) typical infants preferred to watch the mouth rather than the eyes during speech, a preference that reversed with development. These results highlight the importance of taking the effect of development into account when addressing gaze behaviours characteristic of ASD.     

Separate coding of different gaze directions in the superior temporal sulcus and inferior parietal lobule

Electrophysiological recording in the anterior superior temporal sulcus (STS) of monkeys has demonstrated separate cell populations responsive to direct and averted gaze. Human functional imaging has demonstrated posterior STS activation in gaze processing, particularly in coding the intentions conveyed by gaze, but to date has provided no evidence of dissociable coding of different gaze directions. Because the spatial resolution typical of group-based fMRI studies (approximately 6-10 mm) exceeds the size of cellular patches sensitive to different facial characteristics (1-4 mm in monkeys), a more sensitive technique may be required. We therefore used fMRI adaptation, which is considered to offer superior resolution, to investigate whether the human anterior STS contains representations of different gaze directions, as suggested by non-human primate research. Subjects viewed probe faces gazing left, directly ahead, or right. Adapting to leftward gaze produced a reduction in BOLD response to left relative to right (and direct) gaze probes in the anterior STS and inferior parietal cortex; rightward gaze adaptation produced a corresponding reduction to right gaze probes. Consistent with these findings, averted gaze in the adapted direction was misidentified as direct. Our study provides the first human evidence of dissociable neural systems for left and right gaze.     

It’s All in the Eyes: Subcortical and Cortical Activation during Grotesqueness Perception in Autism

Atypical face processing plays a key role in social interaction difficulties encountered by individuals with autism. In the current fMRI study, the Thatcher illusion was used to investigate several aspects of face processing in 20 young adults with high-functioning autism spectrum disorder (ASD) and 20 matched neurotypical controls. “Thatcherized” stimuli were modified at either the eyes or the mouth and participants discriminated between pairs of faces while cued to attend to either of these features in upright and inverted orientation. Behavioral data confirmed sensitivity to the illusion and intact configural processing in ASD. Directing attention towards the eyes vs. the mouth in upright faces in ASD led to (1) improved discrimination accuracy; (2) increased activation in areas involved in social and emotional processing; (3) increased activation in subcortical face-processing areas. Our findings show that when explicitly cued to attend to the eyes, activation of cortical areas involved in face processing, including its social and emotional aspects, can be enhanced in autism. This suggests that impairments in face processing in autism may be caused by a deficit in social attention, and that giving specific cues to attend to the eye-region when performing behavioral therapies aimed at improving social skills may result in a better outcome.     

Facial Expressions of Threat Influence Perceived Gaze Direction in 8 Year-Olds

Adults show reciprocal influences between the perception of gaze direction and emotional expression. These facilitate the understanding of facial signals, because the meaning of one cue can vary considerably depending on the value of the other. Here we ask whether children show similar reciprocal influences in the perception of gaze and expression. A previous study has demonstrated that gaze direction affects the perception of emotional expression in children. Here we demonstrate the opposite direction of influence, showing that expression affects the perception of gaze direction. Specifically, we show that the cone of gaze, i.e., range of gaze deviations perceived as direct, is larger for angry than neutral or fearful faces in 8 year-old children. Therefore, we conclude that children, like adults, show reciprocal influences in the perception of gaze and expression. An unexpected finding was that, compared with adults, children showed larger effects of expression on gaze perception. This finding raises the possibility that it is the ability to process cues independently, rather than sensitivity to combinations, that matures during development. Alternatively, children may be particularly sensitive to anger in adult faces.