On the time course of vocal emotion recognition

How quickly do listeners recognize emotions from a speaker''s voice, and does the time course for recognition vary by emotion type? To address these questions, we adapted the auditory gating paradigm to estimate how much vocal information is needed for listeners to categorize five basic emotions (anger, disgust, fear, sadness, happiness) and neutral utterances produced by male and female speakers of English. Semantically-anomalous pseudo-utterances (e.g., The rivix jolled the silling) conveying each emotion were divided into seven gate intervals according to the number of syllables that listeners heard from sentence onset. Participants (n = 48) judged the emotional meaning of stimuli presented at each gate duration interval, in a successive, blocked presentation format. Analyses looked at how recognition of each emotion evolves as an utterance unfolds and estimated the “identification point” for each emotion. Results showed that anger, sadness, fear, and neutral expressions are recognized more accurately at short gate intervals than happiness, and particularly disgust; however, as speech unfolds, recognition of happiness improves significantly towards the end of the utterance (and fear is recognized more accurately than other emotions). When the gate associated with the emotion identification point of each stimulus was calculated, data indicated that fear (M = 517 ms), sadness (M = 576 ms), and neutral (M = 510 ms) expressions were identified from shorter acoustic events than the other emotions. These data reveal differences in the underlying time course for conscious recognition of basic emotions from vocal expressions, which should be accounted for in studies of emotional speech processing.     

Humans rely on the same rules to assess emotional valence and intensity in conspecific and dog vocalizations

Humans excel at assessing conspecific emotional valence and intensity, based solely on non-verbal vocal bursts that are also common in other mammals. It is not known, however, whether human listeners rely on similar acoustic cues to assess emotional content in conspecific and heterospecific vocalizations, and which acoustical parameters affect their performance. Here, for the first time, we directly compared the emotional valence and intensity perception of dog and human non-verbal vocalizations. We revealed similar relationships between acoustic features and emotional valence and intensity ratings of human and dog vocalizations: those with shorter call lengths were rated as more positive, whereas those with a higher pitch were rated as more intense. Our findings demonstrate that humans rate conspecific emotional vocalizations along basic acoustic rules, and that they apply similar rules when processing dog vocal expressions. This suggests that humans may utilize similar mental mechanisms for recognizing human and heterospecific vocal emotions.     

Human Vocal Attractiveness as Signaled by Body Size Projection

Voice, as a secondary sexual characteristic, is known to affect the perceived attractiveness of human individuals. But the underlying mechanism of vocal attractiveness has remained unclear. Here, we presented human listeners with acoustically altered natural sentences and fully synthetic sentences with systematically manipulated pitch, formants and voice quality based on a principle of body size projection reported for animal calls and emotional human vocal expressions. The results show that male listeners preferred a female voice that signals a small body size, with relatively high pitch, wide formant dispersion and breathy voice, while female listeners preferred a male voice that signals a large body size with low pitch and narrow formant dispersion. Interestingly, however, male vocal attractiveness was also enhanced by breathiness, which presumably softened the aggressiveness associated with a large body size. These results, together with the additional finding that the same vocal dimensions also affect emotion judgment, indicate that humans still employ a vocal interaction strategy used in animal calls despite the development of complex language.     

Expression of Emotional Arousal in Two Different Piglet Call Types

Humans as well as many animal species reveal their emotional state in their voice. Vocal features show strikingly similar correlation patterns with emotional states across mammalian species, suggesting that the vocal expression of emotion follows highly conserved signalling rules. To fully understand the principles of emotional signalling in mammals it is, however, necessary to also account for any inconsistencies in the way that they are acoustically encoded. Here we investigate whether the expression of emotions differs between call types produced by the same species. We compare the acoustic structure of two common piglet calls—the scream (a distress call) and the grunt (a contact call)—across three levels of arousal in a negative situation. We find that while the central frequency of calls increases with arousal in both call types, the amplitude and tonal quality (harmonic-to-noise ratio) show contrasting patterns: as arousal increased, the intensity also increased in screams, but not in grunts, while the harmonicity increased in screams but decreased in grunts. Our results suggest that the expression of arousal depends on the function and acoustic specificity of the call type. The fact that more vocal features varied with arousal in scream calls than in grunts is consistent with the idea that distress calls have evolved to convey information about emotional arousal.     

Vocal communication of simulated pain

While evidence suggests that pain cries produced by human babies and other mammal infants communicate pain intensity, whether the pain vocalisations of human adults also encode pain intensity, and which acoustic characteristics influence listeners’ perceptions, remains unexplored. Here, we investigated how trained actors communicated pain by comparing the acoustic characteristics of nonverbal vocalisations expressing different levels of pain intensity (mild, moderate and severe). We then performed playback experiments to examine whether vocalisers successfully communicated pain intensity to listeners, and which acoustic characteristics were responsible for variation in pain ratings. We found that the mean and range of voice fundamental frequency (F0, perceived as pitch), the amplitude of the vocalisation, the degree of periodicity of the vocalisation and the proportion of the signal displaying non-linear phenomena all increased with the level of simulated pain intensity. In turn, these parameters predicted increases in listeners’ ratings of pain intensity. We also found that while different voice features contributed to increases in pain ratings within each level of expressed pain, a combination of these features explained an impressive amount of the variance in listeners’ pain ratings, both across (76%) and within (31–54%) pain levels. Our results show that adult vocalisers can volitionally simulate and modulate pain vocalisations to influence listeners’ perceptions of pain in a manner consistent with authentic human infant and nonhuman mammal pain vocalisations, and highlight potential for the development of a practical quantitative tool to improve pain assessment in populations unable to self-report their subjective pain experience.     

Cross-Taxa Similarities in Affect-Induced Changes of Vocal Behavior and Voice in Arboreal Monkeys

Measuring the affective state of an individual across species with comparable non-invasive methods is a current challenge in animal communication research. This study aims to explore to which extent affect intensity is conveyed in the vocal behaviours of three nonhuman primate species (Campbell''s monkeys, De Brazza''s monkeys, red-capped mangabeys), which vary in body size, ecological niche and social system. Similarly in the three species, we experimentally induced a change in captive social groups'' affect by locking all group members together in their outside enclosure. The two experimental conditions which varied in affect intensity consisted in imposing a pre-reunion 90 mn-separation by splitting up the respective group into two subgroups (High affect condition) or not (Low affect condition). We measured call rates as well as voice features at the time of reunion in both conditions. The three studied species reacted in a very similar way. Across species, call rates changed significantly between the behaviourally defined states. Furthermore, contact call duration and, to some extent, voice pitch increased. Our results suggest, for the first time in arboreal Old World monkeys, that affect intensity is conveyed reliably in vocal behaviour and specific acoustic characteristics of voice, irrespective of body size and ecological niche differences between species. Cross-taxa similarities in acoustic cues of affect intensity point to phylogenetic constraints and inheritance from a common ancestor, whereas variations in vocal behaviour and affect intensity-related acoustic cues between species may be an adaptation to specific social requirements and depend on social systems. Our findings as well as a comparison with published works on acoustic communication in other vertebrate groups support the hypothesis that affect intensity in human voice originates from precursors already found deep inside the vertebrate phylogeny.     

Women use voice parameters to assess men's characteristics

The purpose of this study was: (i) to provide additional evidence regarding the existence of human voice parameters, which could be reliable indicators of a speaker's physical characteristics and (ii) to examine the ability of listeners to judge voice pleasantness and a speaker's characteristics from speech samples. We recorded 26 men enunciating five vowels. Voices were played to 102 female judges who were asked to assess vocal attractiveness and speakers' age, height and weight. Statistical analyses were used to determine: (i) which physical component predicted which vocal component and (ii) which vocal component predicted which judgment. We found that men with low-frequency formants and small formant dispersion tended to be older, taller and tended to have a high level of testosterone. Female listeners were consistent in their pleasantness judgment and in their height, weight and age estimates. Pleasantness judgments were based mainly on intonation. Female listeners were able to correctly estimate age by using formant components. They were able to estimate weight but we could not explain which acoustic parameters they used. However, female listeners were not able to estimate height, possibly because they used intonation incorrectly. Our study confirms that in all mammal species examined thus far, including humans, formant components can provide a relatively accurate indication of a vocalizing individual's characteristics. Human listeners have the necessary information at their disposal; however, they do not necessarily use it.     

Age- and sex-related variations in vocal-tract morphology and voice acoustics during adolescence

Distinct differences in the human voice emerge during adolescence, with males producing deeper and more resonant voices than females by the end of sexual maturation. Using magnetic resonance images of heads and voice recordings obtained in 532 typically developing adolescents, we investigate what might be the drivers of this change in voice, and the subjective judgment of the voice “maleness” and “femaleness”.We show clear sex differences in the morphology of voice-related structures during adolescence, with males displaying strong associations between age (and puberty) and both vocal-fold and vocal-tract length; this was not the case in female adolescents. At the same time, males (compared with females) display stronger associations between age (and puberty) with both fundamental frequency and formant position. In males, vocal morphology was a mediator in the relationship between bioavailable testosterone and acoustic indices.Subjective judgment of the voice sex could be predicted by the morphological and acoustic parameters in males only: the length of vocal folds and its acoustic counterpart, fundamental frequency, is a larger predictor of subjective “maleness” of a voice than vocal-tract length and formant position.     

Vocal correlates of sender-identity and arousal in the isolation calls of domestic kitten (<Emphasis Type="Italic">Felis silvestris catus</Emphasis>)

Introduction

Human speech does not only communicate linguistic information but also paralinguistic features, e.g. information about the identity and the arousal state of the sender. Comparable morphological and physiological constraints on vocal production in mammals suggest the existence of commonalities encoding sender-identity and the arousal state of a sender across mammals. To explore this hypothesis and to investigate whether specific acoustic parameters encode for sender-identity while others encode for arousal, we studied infants of the domestic cat (Felis silvestris catus). Kittens are an excellent model for analysing vocal correlates of sender-identity and arousal. They strongly depend on the care of their mother. Thus, the acoustical conveyance of sender-identity and arousal may be important for their survival.

Results

We recorded calls of 18 kittens in an experimentally-induced separation paradigm, where kittens were spatially separated from their mother and siblings. In the Low arousal condition, infants were just separated without any manipulation. In the High arousal condition infants were handled by the experimenter. Multi-parametric sound analyses revealed that kitten isolation calls are individually distinct and differ between the Low and High arousal conditions. Our results suggested that source- and filter-related parameters are important for encoding sender-identity, whereas time-, source- and tonality-related parameters are important for encoding arousal.

Conclusion

Comparable findings in other mammalian lineages provide evidence for commonalities in non-verbal cues encoding sender-identity and arousal across mammals comparable to paralinguistic cues in humans. This favours the establishment of general concepts for voice recognition and emotions in humans and animals.

     

Masculine Men Articulate Less Clearly

In previous research, acoustic characteristics of the male voice have been shown to signal various aspects of mate quality and threat potential. But the human voice is also a medium of linguistic communication. The present study explores whether physical and vocal indicators of male mate quality and threat potential are linked to effective communicative behaviors such as vowel differentiation and use of more salient phonetic variants of consonants. We show that physical and vocal indicators of male threat potential, height and formant position, are negatively linked to vowel space size, and that height and levels of circulating testosterone are negatively linked to the use of the aspirated variant of the alveolar stop consonant /t/. Thus, taller, more masculine men display less clarity in their speech and prefer phonetic variants that may be associated with masculine attributes such as toughness. These findings suggest that vocal signals of men’s mate quality and/or dominance are not confined to the realm of voice acoustics but extend to other aspects of communicative behavior, even if this means a trade-off with speech patterns that are considered communicatively advantageous, such as clarity and indexical cues to higher social class.     

The effect of visual cues on difficulty ratings for segregation of musical streams in listeners with impaired hearing

Background

Enjoyment of music is an important part of life that may be degraded for people with hearing impairments, especially those using cochlear implants. The ability to follow separate lines of melody is an important factor in music appreciation. This ability relies on effective auditory streaming, which is much reduced in people with hearing impairment, contributing to difficulties in music appreciation. The aim of this study was to assess whether visual cues could reduce the subjective difficulty of segregating a melody from interleaved background notes in normally hearing listeners, those using hearing aids, and those using cochlear implants.

Methodology/Principal Findings

Normally hearing listeners (N = 20), hearing aid users (N = 10), and cochlear implant users (N = 11) were asked to rate the difficulty of segregating a repeating four-note melody from random interleaved distracter notes. The pitch of the background notes was gradually increased or decreased throughout blocks, providing a range of difficulty from easy (with a large pitch separation between melody and distracter) to impossible (with the melody and distracter completely overlapping). Visual cues were provided on half the blocks, and difficulty ratings for blocks with and without visual cues were compared between groups. Visual cues reduced the subjective difficulty of extracting the melody from the distracter notes for normally hearing listeners and cochlear implant users, but not hearing aid users.

Conclusion/Significance

Simple visual cues may improve the ability of cochlear implant users to segregate lines of music, thus potentially increasing their enjoyment of music. More research is needed to determine what type of acoustic cues to encode visually in order to optimise the benefits they may provide.     

Ultrasonic vocalizations of adult male Foxp2‐mutant mice: behavioral contexts of arousal and emotion

Adult mouse ultrasonic vocalizations (USVs) occur in multiple behavioral and stimulus contexts associated with various levels of arousal, emotion and social interaction. Here, in three experiments of increasing stimulus intensity (water; female urine; male interacting with adult female), we tested the hypothesis that USVs of adult males express the strength of arousal and emotion via different USV parameters (18 parameters analyzed). Furthermore, we analyzed two mouse lines with heterozygous Foxp2 mutations (R552H missense, S321X nonsense), known to produce severe speech and language disorders in humans. These experiments allowed us to test whether intact Foxp2 function is necessary for developing full adult USV repertoires, and whether mutations of this gene influence instinctive vocal expressions based on arousal and emotion. The results suggest that USV calling rate characterizes the arousal level, while sound pressure and spectrotemporal call complexity (overtones/harmonics, type of frequency jumps) may provide indices of levels of positive emotion. The presence of Foxp2 mutations did not qualitatively affect the USVs; all USV types that were found in wild‐type animals also occurred in heterozygous mutants. However, mice with Foxp2 mutations displayed quantitative differences in USVs as compared to wild‐types, and these changes were context dependent. Compared to wild‐type animals, heterozygous mutants emitted mainly longer and louder USVs at higher minimum frequencies with a higher occurrence rate of overtones/harmonics and complex frequency jump types. We discuss possible hypotheses about Foxp2 influence on emotional vocal expressions, which can be investigated in future experiments using selective knockdown of Foxp2 in specific brain circuits.     

Context-related acoustic variation in male fallow deer (Dama dama) groans

While social and behavioural contexts are known to affect the acoustic structure of vocal signals in several mammal species, few studies have investigated context-related acoustic variation during inter-sexual advertisement and/or intra-sexual competition. Here we recorded male fallow deer groans during the breeding season and investigated how key acoustic parameters (fundamental frequency and formant frequencies) vary as a function of the social context in which they are produced. We found that in the presence of females, male fallow deer produced groans with higher mean fundamental frequency when vocal males were also present than they did when no vocal males were in close vicinity. We attribute this to the increased arousal state typically associated with this context. In addition, groan minimum formant frequency spacing was slightly, but significantly lower (indicating marginally more extended vocal tracts) when males were alone than when potential mates and/or competitors were nearby. This indicates that, contrary to our predictions, male fallow deer do not exaggerate the acoustic impression of their body size by further lowering their formant frequencies in the presence of potential mating partners and competitors. Furthermore, since the magnitude of the variation in groan minimum formant frequency spacing remains small compared to documented inter-individual differences, our findings are consistent with the hypothesis that formants are reliable static cues to body size during intra- and inter-sexual advertisement that do not concurrently encode dynamic motivation-related information.     

Vocal cues of ovulation in human females

Recent research has documented a variety of ovulatory cues in humans, and in many nonhuman species, the vocal channel provides cues of reproductive state. We collected two sets of vocal samples from 69 normally ovulating women: one set during the follicular (high-fertility) phase of the cycle and one set during the luteal (low-fertility) phase, with ovulation confirmed by luteinizing hormone tests. In these samples we measured fundamental frequency (pitch), formant dispersion, jitter, shimmer, harmonics-to-noise ratio and speech rate. When speaking a simple introductory sentence, women's pitch increased during high- as compared with low-fertility, and this difference was the greatest for women whose voices were recorded on the two highest fertility days within the fertile window (the 2 days just before ovulation). This pattern did not occur when the same women produced vowels. The high- versus low-fertility difference in pitch was associated with the approach of ovulation and not menstrual onset, thus representing, to our knowledge, the first research to show a specific cyclic fertility cue in the human voice. We interpret this finding as evidence of a fertility-related enhancement of femininity consistent with other research documenting attractiveness-related changes associated with ovulation.     

Vocal-tract resonances as indexical cues in rhesus monkeys

Vocal-tract resonances (or formants) are acoustic signatures in the voice and are related to the shape and length of the vocal tract. Formants play an important role in human communication, helping us not only to distinguish several different speech sounds [1], but also to extract important information related to the physical characteristics of the speaker, so-called indexical cues. How did formants come to play such an important role in human vocal communication? One hypothesis suggests that the ancestral role of formant perception--a role that might be present in extant nonhuman primates--was to provide indexical cues [2-5]. Although formants are present in the acoustic structure of vowel-like calls of monkeys [3-8] and implicated in the discrimination of call types [8-10], it is not known whether they use this feature to extract indexical cues. Here, we investigate whether rhesus monkeys can use the formant structure in their "coo" calls to assess the age-related body size of conspecifics. Using a preferential-looking paradigm [11, 12] and synthetic coo calls in which formant structure simulated an adult/large- or juvenile/small-sounding individual, we demonstrate that untrained monkeys attend to formant cues and link large-sounding coos to large faces and small-sounding coos to small faces-in essence, they can, like humans [13], use formants as indicators of age-related body size.     

Why Do Large Dogs Sound More Aggressive to Human Listeners: Acoustic Bases of Motivational Misattributions

Previous research has highlighted that while human listeners are capable of estimating the body size of dogs using the acoustic components of their growls, they also rate growls from larger dogs as more being aggressive than growls from smaller dogs. The aim of this study was to investigate the relative contributions of two cues to body size, fundamental frequency (F0) and formant frequency dispersion (Δf) to perceived levels of aggression. We found that participants that had just made an accurate assessment of caller size based on these characteristics then misattributed aggressiveness levels on the basis of these same size‐related acoustic cues. More specifically, stimuli in which F0 and/or Δf were typical of larger dogs were rated as being more aggressive than stimuli typical of smaller dogs. Although both F0 and Δf influenced aggressiveness ratings independently, their interaction also had a significant affect. These results are discussed with respects to the human tendency to generalise reliable between‐class acoustic cues to within‐class stimuli and the resulting potential for making perceptual misattributions.     

A Bayesian decision analysis approach to assess voice disorder risks by using acoustic features

Vocal fold nodules are recognized as an occupational disease for all collective of workers performing activities for which maintained and continued use of voice is required. Computer‐aided systems based on features extracted from voice recordings have been considered as potential noninvasive and low cost tools to diagnose some voice‐related diseases. A Bayesian decision analysis approach has been proposed to classify university lectures in three levels of risk: low, medium, and high, based on the information provided by acoustic features extracted from healthy controls and people suffering from vocal fold nodules. The proposed risk groups are associated with different treatments. The approach is based on the calculation of posterior probabilities of developing vocal fold nodules and considers utility functions that include the financial cost and the probability of recovery for the corresponding treatment. Maximization of the expected utilities is considered. By using this approach, the risk of having vocal fold nodules is identified for each university lecturer, so he/she can be properly assigned to the right treatment. The approach has been applied to university lecturers according to the Disease Prevention Program of the University of Extremadura. However, it can also be applied to other voice professionals (singers, speakers, coaches, actors…).     

Are dogs able to recognize their handler's voice? A preliminary study

Abstract

A number of studies have dealt with the ability of dogs to rely on olfactory cues to recognize their owner or handler. However, in the absence of these cues (e.g., when working at a distance from their owner) dogs have to use other senses to interact with their owner. Considering the high sensitivity of their auditory system, we can hypothesize that dogs are able to use vocal cues for this purpose. Certainly, it is commonly assumed that dogs can discriminate between people on the basis of their voice, and yet there is a clear lack of empirical data on this subject. The present study assesses the importance of the handler's olfactory and visual cues versus the importance of his/her vocal cues to the working behavior of dogs. I compare the ability of working dogs to obey different vocal commands issued either by their handler (HV) or by a stranger (SV) via loudspeakers in different olfactory contexts: presence of their handler (H) or of a stranger (S). The dogs' performance (percentage of correct responses) decreased when the acoustic cues were modified (H+HV tended to differ from the H+SV—p = 0.09—and differed from the S+SV conditions—p = 0.05) but not when only the olfactory cues were changed (no difference between H+HV and S+HV conditions—p > 0.1). Consequently, we can assume that dogs rely, at least in part, on acoustic parameters of voices to recognize and cooperate with their handler.     

Do Women's Voices Provide Cues of the Likelihood of Ovulation? The Importance of Sampling Regime

The human voice provides a rich source of information about individual attributes such as body size, developmental stability and emotional state. Moreover, there is evidence that female voice characteristics change across the menstrual cycle. A previous study reported that women speak with higher fundamental frequency (F0) in the high-fertility compared to the low-fertility phase. To gain further insights into the mechanisms underlying this variation in perceived attractiveness and the relationship between vocal quality and the timing of ovulation, we combined hormone measurements and acoustic analyses, to characterize voice changes on a day-to-day basis throughout the menstrual cycle. Voice characteristics were measured from free speech as well as sustained vowels. In addition, we asked men to rate vocal attractiveness from selected samples. The free speech samples revealed marginally significant variation in F0 with an increase prior to and a distinct drop during ovulation. Overall variation throughout the cycle, however, precluded unequivocal identification of the period with the highest conception risk. The analysis of vowel samples revealed a significant increase in degree of unvoiceness and noise-to-harmonic ratio during menstruation, possibly related to an increase in tissue water content. Neither estrogen nor progestogen levels predicted the observed changes in acoustic characteristics. The perceptual experiments revealed a preference by males for voice samples recorded during the pre-ovulatory period compared to other periods in the cycle. While overall we confirm earlier findings in that women speak with a higher and more variable fundamental frequency just prior to ovulation, the present study highlights the importance of taking the full range of variation into account before drawing conclusions about the value of these cues for the detection of ovulation.     

Integrating cues of social interest and voice pitch in men's preferences for women's voices

Most previous studies of vocal attractiveness have focused on preferences for physical characteristics of voices such as pitch. Here we examine the content of vocalizations in interaction with such physical traits, finding that vocal cues of social interest modulate the strength of men's preferences for raised pitch in women's voices. Men showed stronger preferences for raised pitch when judging the voices of women who appeared interested in the listener than when judging the voices of women who appeared relatively disinterested in the listener. These findings show that voice preferences are not determined solely by physical properties of voices and that men integrate information about voice pitch and the degree of social interest expressed by women when forming voice preferences. Women's preferences for raised pitch in women's voices were not modulated by cues of social interest, suggesting that the integration of cues of social interest and voice pitch when men judge the attractiveness of women's voices may reflect adaptations that promote efficient allocation of men's mating effort.