Adaptation Aftereffects in Vocal Emotion Perception Elicited by Expressive Faces and Voices

The perception of emotions is often suggested to be multimodal in nature, and bimodal as compared to unimodal (auditory or visual) presentation of emotional stimuli can lead to superior emotion recognition. In previous studies, contrastive aftereffects in emotion perception caused by perceptual adaptation have been shown for faces and for auditory affective vocalization, when adaptors were of the same modality. By contrast, crossmodal aftereffects in the perception of emotional vocalizations have not been demonstrated yet. In three experiments we investigated the influence of emotional voice as well as dynamic facial video adaptors on the perception of emotion-ambiguous voices morphed on an angry-to-happy continuum. Contrastive aftereffects were found for unimodal (voice) adaptation conditions, in that test voices were perceived as happier after adaptation to angry voices, and vice versa. Bimodal (voice + dynamic face) adaptors tended to elicit larger contrastive aftereffects. Importantly, crossmodal (dynamic face) adaptors also elicited substantial aftereffects in male, but not in female participants. Our results (1) support the idea of contrastive processing of emotions (2), show for the first time crossmodal adaptation effects under certain conditions, consistent with the idea that emotion processing is multimodal in nature, and (3) suggest gender differences in the sensory integration of facial and vocal emotional stimuli.     

Perceptual auditory aftereffects on voice identity using brief vowel stimuli

Humans can identify individuals from their voice, suggesting the existence of a perceptual representation of voice identity. We used perceptual aftereffects - shifts in perceived stimulus quality after brief exposure to a repeated adaptor stimulus - to further investigate the representation of voice identity in two experiments. Healthy adult listeners were familiarized with several voices until they reached a recognition criterion. They were then tested on identification tasks that used vowel stimuli generated by morphing between the different identities, presented either in isolation (baseline) or following short exposure to different types of voice adaptors (adaptation). Experiment 1 showed that adaptation to a given voice induced categorization shifts away from that adaptor's identity even when the adaptors consisted of vowels different from the probe stimuli. Moreover, original voices and caricatures resulted in comparable aftereffects, ruling out an explanation of identity aftereffects in terms of adaptation to low-level features. In Experiment 2, we show that adaptors with a disrupted configuration, i.e., altered fundamental frequency or formant frequencies, failed to produce perceptual aftereffects showing the importance of the preserved configuration of these acoustical cues in the representation of voices. These two experiments indicate a high-level, dynamic representation of voice identity based on the combination of several lower-level acoustical features into a specific voice configuration.     

Implicit multisensory associations influence voice recognition

Natural objects provide partially redundant information to the brain through different sensory modalities. For example, voices and faces both give information about the speech content, age, and gender of a person. Thanks to this redundancy, multimodal recognition is fast, robust, and automatic. In unimodal perception, however, only part of the information about an object is available. Here, we addressed whether, even under conditions of unimodal sensory input, crossmodal neural circuits that have been shaped by previous associative learning become activated and underpin a performance benefit. We measured brain activity with functional magnetic resonance imaging before, while, and after participants learned to associate either sensory redundant stimuli, i.e. voices and faces, or arbitrary multimodal combinations, i.e. voices and written names, ring tones, and cell phones or brand names of these cell phones. After learning, participants were better at recognizing unimodal auditory voices that had been paired with faces than those paired with written names, and association of voices with faces resulted in an increased functional coupling between voice and face areas. No such effects were observed for ring tones that had been paired with cell phones or names. These findings demonstrate that brief exposure to ecologically valid and sensory redundant stimulus pairs, such as voices and faces, induces specific multisensory associations. Consistent with predictive coding theories, associative representations become thereafter available for unimodal perception and facilitate object recognition. These data suggest that for natural objects effective predictive signals can be generated across sensory systems and proceed by optimization of functional connectivity between specialized cortical sensory modules.     

Adding Years to Your Life (or at Least Looking Like It): A Simple Normalization Underlies Adaptation to Facial Age

Adaptation has been widely used to probe how experience shapes the visual encoding of faces, but the pattern of perceptual changes produced by adaptation and the neural mechanisms these imply remain poorly characterized. We explored how adaptation alters the perceived age of faces, a fundamental facial attribute which can uniquely and reliably be scaled by observers. This allowed us to measure how adaptation to one age level affected the full continuum of perceived ages. Participants guessed the ages of faces ranging from 18–89, before or after adapting to a different set of faces composed of younger, older, or middle-aged adults. Adapting to young or old faces induced opposite linear shifts in perceived age that were independent of the model''s age. Specifically, after adapting to younger or older faces, faces of all ages appeared 2 to 3 years older or younger, respectively. In contrast, middle-aged adaptors induced no aftereffects. This pattern suggests that adaptation leads to a simple and uniform renormalization of age perception, and is consistent with a norm-based neural code for the mechanisms mediating the perception of facial age.     

Vocal and visual attractiveness are related in women

We investigated the relation between visual and vocal attractiveness in women as judged by men. We recorded 34 women speaking four vowels and measured the peak frequency, the first five harmonic frequencies, the first five formant frequencies and formant dispersion. The women were also photographed (head shot), several body measures were taken and their ages were recorded. The voices were played to male judges who were asked to assess the women's age and vocal attractiveness from the recording. The men were then asked to assess the attractiveness of the photographs. Men were in strong agreement on which was an attractive voice and face; and women with attractive faces had attractive voices. Higher-frequency voices were assessed as being more attractive and as belonging to younger women (the lowest frequency produced is a good indicator of age in women in general). Larger women had lower voices and were judged as having less attractive faces and voices. Taller women had narrower formant dispersion as predicted. The results imply that different measures of attractiveness are in agreement and signal similar qualities, such as female age, body size and possibly hormonal profile. Copyright 2003 Published by Elsevier Science Ltd on behalf of The Association for the Study of Animal Behaviour.      

"Putting the face to the voice": matching identity across modality

Speech perception provides compelling examples of a strong link between auditory and visual modalities. This link originates in the mechanics of speech production, which, in shaping the vocal tract, determine the movement of the face as well as the sound of the voice. In this paper, we present evidence that equivalent information about identity is available cross-modally from both the face and voice. Using a delayed matching to sample task, XAB, we show that people can match the video of an unfamiliar face, X, to an unfamiliar voice, A or B, and vice versa, but only when stimuli are moving and are played forward. The critical role of time-varying information is underlined by the ability to match faces to voices containing only the coarse spatial and temporal information provided by sine wave speech [5]. The effect of varying sentence content across modalities was small, showing that identity-specific information is not closely tied to particular utterances. We conclude that the physical constraints linking faces to voices result in bimodally available dynamic information, not only about what is being said, but also about who is saying it.     

Adaptor Identity Modulates Adaptation Effects in Familiar Face Identification and Their Neural Correlates

Adaptation-related aftereffects (AEs) show how face perception can be altered by recent perceptual experiences. Along with contrastive behavioural biases, modulations of the early event-related potentials (ERPs) were typically reported on categorical levels. Nevertheless, the role of the adaptor stimulus per se for face identity-specific AEs is not completely understood and was therefore investigated in the present study. Participants were adapted to faces (S1s) varying systematically on a morphing continuum between pairs of famous identities (identities A and B), or to Fourier phase-randomized faces, and had to match the subsequently presented ambiguous faces (S2s; 50/50% identity A/B) to one of the respective original faces. We found that S1s identical with or near to the original identities led to strong contrastive biases with more identity B responses following A adaptation and vice versa. In addition, the closer S1s were to the 50/50% S2 on the morphing continuum, the smaller the magnitude of the AE was. The relation between S1s and AE was, however, not linear. Additionally, stronger AEs were accompanied by faster reaction times. Analyses of the simultaneously recorded ERPs revealed categorical adaptation effects starting at 100 ms post-stimulus onset, that were most pronounced at around 125–240 ms for occipito-temporal sites over both hemispheres. S1-specific amplitude modulations were found at around 300–400 ms. Response-specific analyses of ERPs showed reduced voltages starting at around 125 ms when the S1 biased perception in a contrastive way as compared to when it did not. Our results suggest that face identity AEs do not only depend on physical differences between S1 and S2, but also on perceptual factors, such as the ambiguity of S1. Furthermore, short-term plasticity of face identity processing might work in parallel to object-category processing, and is reflected in the first 400 ms of the ERP.     

Gender in facial representations: a contrast-based study of adaptation within and between the sexes

Face aftereffects are proving to be an effective means of examining the properties of face-specific processes in the human visual system. We examined the role of gender in the neural representation of faces using a contrast-based adaptation method. If faces of different genders share the same representational face space, then adaptation to a face of one gender should affect both same- and different-gender faces. Further, if these aftereffects differ in magnitude, this may indicate distinct gender-related factors in the organization of this face space. To control for a potential confound between physical similarity and gender, we used a Bayesian ideal observer and human discrimination data to construct a stimulus set in which pairs of different-gender faces were equally dissimilar as same-gender pairs. We found that the recognition of both same-gender and different-gender faces was suppressed following a brief exposure of 100 ms. Moreover, recognition was more suppressed for test faces of a different-gender than those of the same-gender as the adaptor, despite the equivalence in physical and psychophysical similarity. Our results suggest that male and female faces likely occupy the same face space, allowing transfer of aftereffects between the genders, but that there are special properties that emerge along gender-defining dimensions of this space.     

不同物种雄性鸣声刺激下雌性凹耳蛙脑组织中ZENK的表达

通过免疫组化SABC法,研究在背景噪音刺激、雄性凹耳蛙(Odorrana tormota)声音刺激以及雄性大绿臭蛙(O. graminea)声音刺激下ZENK蛋白在雌性凹耳蛙脑部的表达。结果显示,上述三种声音刺激下雌性凹耳蛙的端脑、间脑和小脑区域均未发现ZENK蛋白的阳性细胞。在雄性凹耳蛙声音刺激下,雌性凹耳蛙中脑的视顶盖、脚间核、中脑深部核、被盖、半环隆枕以及延脑的孤束核、中缝核、网状核、网状结构、上橄榄核部位均有阳性细胞分布,在大绿臭蛙声音刺激下,这些部位的阳性细胞数量明显减少。本实验中ZENK蛋白阳性细胞表达部位的一部分是位于与两栖类处理听觉信息有关的中脑半环隆枕、被盖以及延脑的上橄榄核,由此可推断,在雌性凹耳蛙ZENK蛋白与听觉信息处理有着密不可分的联系。     

Imagine Jane and identify John: face identity aftereffects induced by imagined faces

It is not known whether prolonged exposure to perceived and imagined complex visual images produces similar shifts in subsequent perception through selective adaptation. This question is important because a positive finding would suggest that perception and imagery of visual stimuli are mediated by shared neural networks. In this study, we used a selective adaptation procedure designed to induce high-level face-identity aftereffects--a phenomenon in which extended exposure to a particular face facilitates recognition of subsequent faces with opposite features while impairing recognition of all other faces. We report here that adaptation to either real or imagined faces produces a similar shift in perception and that identity boundaries represented in real and imagined faces are equivalent. Together, our results show that identity information contained in imagined and real faces produce similar behavioral outcomes. Our findings of high-level visual aftereffects induced by imagined stimuli can be taken as evidence for the involvement of shared neural networks that mediate perception and imagery of complex visual stimuli.     

A Comparison of Sensorimotor Adaptation in the Visual and in the Auditory Modality

We compared sensorimotor adaptation in the visual and the auditory modality. Subjects pointed to visual targets while receiving direct spatial information about fingertip position in the visual modality, or they pointed to visual targets while receiving indirect information about fingertip position in the visual modality, or they pointed to auditory targets while receiving indirect information about fingertip position in the auditory modality. Feedback was laterally shifted to induce adaptation, and aftereffects were tested with both target modalities and both hands. We found that aftereffects of adaptation were smaller when tested with the non-adapted hand, i.e., intermanual transfer was incomplete. Furthermore, aftereffects were smaller when tested in the non-adapted target modality, i.e., intermodal transfer was incomplete. Aftereffects were smaller following adaptation with indirect rather than direct feedback, but they were not smaller following adaptation with auditory rather than visual targets. From this we conclude that the magnitude of adaptive recalibration rather depends on the method of feedback delivery (indirect versus direct) than on the modality of feedback (visual versus auditory).     

Preferences for very low and very high voice pitch in humans

Manipulations of voice pitch have been shown to alter attractiveness ratings, but whether preferences extend to very low or very high voice pitch is unknown. Here, we manipulated voice pitch in averaged men's and women's voices by 2 Hz intervals to create a range of male and female voices speaking monopthong vowel sounds and spanning a range of frequencies from normal to very low and very high pitch. With these voices, we used the method of constant stimuli to measure preferences for voice. Nineteen university students (ages: 20-25) participated in three experiments. On average, men preferred high-pitched women's voices to low-pitched women's voices across all frequencies tested. On average, women preferred men's voices lowered in pitch, but did not prefer very low men's voices. The results of this study may reflect selection pressures for men's and women's voices, and shed light on a perceptual link between voice pitch and vocal attractiveness.     

Effects of Awareness on Numerosity Adaptation

Numerosity perception is a process involving several stages of visual processing. This study investigated whether distinct mechanisms exist in numerosity adaptation under different awareness conditions to characterize how numerosity perception occurs at each stage. The status of awareness was controlled by masking conditions, in which monoptic and dichoptic masking were proposed to influence different levels of processing. Numerosity adaptation showed significant aftereffects when the participants were aware (monoptic masking) and unaware (dichoptic masking) of adaptors. The interocular transfer for numerosity adaptation was distinct under the different awareness conditions. Adaptation was primarily binocular when participants were aware of stimuli and was purely monocular when participants were unaware of adaptors. Moreover, numerosity adaptation was significantly reduced when the adaptor dots were clustered into chunks with awareness, whereas clustering had no effect on unaware adaptation. These results show that distinct mechanisms exist in numerosity processing under different awareness conditions. It is suggested that awareness is crucial to numerosity cognition. With awareness, grouping (by clustering) influences numerosity coding through altered object representations, which involves higher-level cognitive processing.     

Effect of Formant Frequency Spacing on Perceived Gender in Pre-Pubertal Children's Voices

Background

It is usually possible to identify the sex of a pre-pubertal child from their voice, despite the absence of sex differences in fundamental frequency at these ages. While it has been suggested that the overall spacing between formants (formant frequency spacing - ΔF) is a key component of the expression and perception of sex in children''s voices, the effect of its continuous variation on sex and gender attribution has not yet been investigated.

Methodology/Principal findings

In the present study we manipulated voice ΔF of eight year olds (two boys and two girls) along continua covering the observed variation of this parameter in pre-pubertal voices, and assessed the effect of this variation on adult ratings of speakers'' sex and gender in two separate experiments. In the first experiment (sex identification) adults were asked to categorise the voice as either male or female. The resulting identification function exhibited a gradual slope from male to female voice categories. In the second experiment (gender rating), adults rated the voices on a continuum from “masculine boy” to “feminine girl”, gradually decreasing their masculinity ratings as ΔF increased.

Conclusions/Significance

These results indicate that the role of ΔF in voice gender perception, which has been reported in adult voices, extends to pre-pubertal children''s voices: variation in ΔF not only affects the perceived sex, but also the perceived masculinity or femininity of the speaker. We discuss the implications of these observations for the expression and perception of gender in children''s voices given the absence of anatomical dimorphism in overall vocal tract length before puberty.     

Voice cells in the primate temporal lobe

Communication signals are important for social interactions and survival and are thought to receive specialized processing in the visual and auditory systems. Whereas the neural processing of faces by face clusters and face cells has been repeatedly studied [1-5], less is known about the neural representation of voice content. Recent functional magnetic resonance imaging (fMRI) studies have localized voice-preferring regions in the primate temporal lobe [6, 7], but the hemodynamic response cannot directly assess neurophysiological properties. We investigated the responses of neurons in an fMRI-identified voice cluster in awake monkeys, and here we provide the first systematic evidence for voice cells. "Voice cells" were identified, in analogy to "face cells," as neurons responding at least 2-fold stronger to conspecific voices than to "nonvoice" sounds or heterospecific voices. Importantly, whereas face clusters are thought to contain high proportions of face cells [4] responding broadly to many faces [1, 2, 4, 5, 8-10], we found that voice clusters contain moderate proportions of voice cells. Furthermore, individual voice cells exhibit high stimulus selectivity. The results reveal the neurophysiological bases for fMRI-defined voice clusters in the primate brain and highlight potential differences in how the auditory and?visual systems generate selective representations of communication signals.     

Female Preferences for Male Vocal and Facial Masculinity in Videos

Vocal and facial masculinity are cues to underlying testosterone in men and influence women’s mate preferences. Consistent with the proposal that facial and vocal masculinity signal common information about men, prior work has revealed correlated female preferences for male facial and vocal masculinity. Previous studies have assessed women’s preferences for male facial and vocal masculinity by presenting faces and voices independently and using static face stimuli. By contrast, here we presented women with short video clips in which male faces and voices were simultaneously manipulated in masculinity. We found that women who preferred masculine faces also preferred masculine voices. Furthermore, women whose faces were rated as relatively more attractive preferred both facial and vocal masculinity more than did women whose faces were rated as less attractive. These findings complement other evidence for cross‐modal masculinity preferences among women and demonstrate that preferences observed in studies using still images and/or independently presented vocal stimuli are also observed when dynamic faces and voices are displayed simultaneously in video format.     

Men's Preferences for Women's Femininity in Dynamic Cross-Modal Stimuli

Men generally prefer feminine women''s faces and voices over masculine women''s faces and voices, and these cross-modal preferences are positively correlated. Men''s preferences for female facial and vocal femininity have typically been investigated independently by presenting soundless still images separately from audio-only vocal recordings. For the first time ever, we presented men with short video clips in which dynamic faces and voices were simultaneously manipulated in femininity/masculinity. Men preferred feminine men''s faces over masculine men''s faces, and preferred masculine men''s voices over feminine men''s voices. We found that men preferred feminine women''s faces and voices over masculine women''s faces and voices. Men''s attractiveness ratings of both feminine and masculine faces were increased by the addition of vocal femininity. Also, men''s attractiveness ratings of feminine and masculine voices were increased by the addition of facial femininity present in the video. Men''s preferences for vocal and facial femininity were significantly and positively correlated when stimuli were female, but not when they were male. Our findings complement other evidence for cross-modal femininity preferences among male raters, and show that preferences observed in studies using still images and/or independently presented vocal stimuli are also observed when dynamic faces and voices are displayed simultaneously in video format.     

The influence of voice pitch on perceptions of trustworthiness across social contexts

Perceptions of trustworthiness are an important predictor of social outcomes, such as monetary exchanges, criminal sentencing, and the attainment of leadership roles. Higher testosterone levels predict both lower voice pitch and untrustworthy behavior, across economic and mating contexts. Here, we tested the influence of voice pitch on perceptions of trustworthiness across general, economic, and mating-related (mate poaching, infidelity) contexts. We found that the context of trust and the sex of the speaker both changed how voice pitch affected perceived trustworthiness. Listeners were more trusting of higher-pitched female voices in economic and mate poaching contexts, but trusted lower-pitched female voices more in general. Listeners were more trusting of higher-pitched male voices in economic and mating-related contexts, and also tended to perceive higher-pitched male voices as more trustworthy in general. Listeners' attributions of trustworthiness were generally unrelated to perceptions of attractiveness from similarly-pitched voices, indicating that trust-related attributions were independent of preferences for higher- or lower-pitched voices. Furthermore, perceptions of general trustworthiness were associated with perceptions of economic trust, but were not consistently associated with perceptions of mating-related trust. These findings provide evidence that voice pitch alone is sufficient to influence trust-related perceptions, and demonstrates that listeners use voice pitch as a cue to trustworthy behavior.     

Cross-Category Adaptation: Objects Produce Gender Adaptation in the Perception of Faces

Adaptation aftereffects have been found for low-level visual features such as colour, motion and shape perception, as well as higher-level features such as gender, race and identity in domains such as faces and biological motion. It is not yet clear if adaptation effects in humans extend beyond this set of higher order features. The aim of this study was to investigate whether objects highly associated with one gender, e.g. high heels for females or electric shavers for males can modulate gender perception of a face. In two separate experiments, we adapted subjects to a series of objects highly associated with one gender and subsequently asked participants to judge the gender of an ambiguous face. Results showed that participants are more likely to perceive an ambiguous face as male after being exposed to objects highly associated to females and vice versa. A gender adaptation aftereffect was obtained despite the adaptor and test stimuli being from different global categories (objects and faces respectively). These findings show that our perception of gender from faces is highly affected by our environment and recent experience. This suggests two possible mechanisms: (a) that perception of the gender associated with an object shares at least some brain areas with those responsible for gender perception of faces and (b) adaptation to gender, which is a high-level concept, can modulate brain areas that are involved in facial gender perception through top-down processes.     

Perceptions of Competence,Strength, and Age Influence Voters to Select Leaders with Lower-Pitched Voices

Voters prefer leaders with lower-pitched voices because they are perceived as stronger, having greater physical prowess, more competent, and having greater integrity. An alternative hypothesis that has yet to be tested is that lower-pitched voices are perceived as older and thus wiser and more experienced. Here the relationships between candidate voice pitch, candidate age, and electoral success are examined with two experiments. Study 1 tests whether voters discriminate on candidate age. The results show that male and female candidates in their 40s and 50s, the time in the lifecycle when voice pitch is at its lowest, are preferred over candidates in their 30s, 60s, and 70s. Study 2 shows that the preference for leaders with lower-pitched voices correlates with the perception that speakers with lower voices are stronger, more competent, and older, but the influence of perception of age on vote choice is the weakest of the three.