Perception of emotional intonation of noisy speech signal with different acoustic parameters by adults of different age and gender

The listener-distinctive features of recognition of different emotional intonations (positive, negative and neutral) of male and female speakers in the presence or absence of background noise were studied in 49 adults aged 20-79 years. In all the listeners noise produced the most pronounced decrease in recognition accuracy for positive emotional intonation ("joy") as compared to other intonations, whereas it did not influence the recognition accuracy of "anger" in 65-79-year-old listeners. The higher emotion recognition rates of a noisy signal were observed for speech emotional intonations expressed by female speakers. Acoustic characteristics of noisy and clear speech signals underlying perception of speech emotional prosody were found for adult listeners of different age and gender.     

Specialization of cerebral hemispheres in the perception of Russian intonations

Relations between the brain hemispheres were studied during the human perception of various types of Russian intonations. Fifty healthy subjects with normal hearing took part in the tests based on the method of monaural presentation of stimuli—the sentences that represented the main kinds of Russian emotional and linguistic intonations. The linguistic intonations expressed: various communicative types of sentences; completeness/incompleteness of a statement; various types of the syntagmatic segmentation of the statements; various logical stress. Sentences that required the identification of the emotion quality were used to study the perception of emotional intonations. The results of statistical analysis of latent periods and errors made by the test subjects demonstrated a significant preference of theright hemisphere in perceiving emotional intonations and complete/incomplete sentences; sentences with different logical stress were perceived mainly by theleft hemisphere. No significant differences were found in the perception of various communicative types of sentences and statements with different syntagmatic segmentation. The obtained data also testify to a difference in the degree of the involvement of human hemispheres in the perception and analysis of prosodic characteristics of the speech in males and females.     

Perception of the emotional component of speech by stuttering children against the background of noise: I. Analysis of the efficiency of the identification of various emotions

The study was devoted to the perception of the speech emotional component by stuttering children under noise impact conditions. The method of study was the evaluation of the probability of an accurate identification of various emotions. Stuttering children were found to be less efficient in identifying all emotions. This fact permitted the assumption that the mechanisms ensuring the identification of emotions against the background of noise by stuttering children form in ontogeny later than that in normally speaking children. Interference immunity of the perception of emotions depends on the emotional coloration of speech. The interhemispheric relations found during the perception of emotions are unstable and, when speech is masked by a noise, acquire the direction that is characteristic of normal children. Thus, the detected ontogenetic features permit one to assume that the establishment of the pattern of interhemispheric relations that is characteristic of normal children is among the reasons of the weakening of stuttering against the background of noise under such conditions.     

Working Memory Is Partially Preserved during Sleep

Although several cognitive processes, including speech processing, have been studied during sleep, working memory (WM) has never been explored up to now. Our study assessed the capacity of WM by testing speech perception when the level of background noise and the sentential semantic length (SSL) (amount of semantic information required to perceive the incongruence of a sentence) were modulated. Speech perception was explored with the N400 component of the event-related potentials recorded to sentence final words (50% semantically congruent with the sentence, 50% semantically incongruent). During sleep stage 2 and paradoxical sleep: (1) without noise, a larger N400 was observed for (short and long SSL) sentences ending with a semantically incongruent word compared to a congruent word (i.e. an N400 effect); (2) with moderate noise, the N400 effect (observed at wake with short and long SSL sentences) was attenuated for long SSL sentences. Our results suggest that WM for linguistic information is partially preserved during sleep with a smaller capacity compared to wake.     

Common Premotor Regions for the Perception and Production of Prosody and Correlations with Empathy and Prosodic Ability

Background

Prosody, the melody and intonation of speech, involves the rhythm, rate, pitch and voice quality to relay linguistic and emotional information from one individual to another. A significant component of human social communication depends upon interpreting and responding to another person''s prosodic tone as well as one''s own ability to produce prosodic speech. However there has been little work on whether the perception and production of prosody share common neural processes, and if so, how these might correlate with individual differences in social ability.

Methods

The aim of the present study was to determine the degree to which perception and production of prosody rely on shared neural systems. Using fMRI, neural activity during perception and production of a meaningless phrase in different prosodic intonations was measured. Regions of overlap for production and perception of prosody were found in premotor regions, in particular the left inferior frontal gyrus (IFG). Activity in these regions was further found to correlate with how high an individual scored on two different measures of affective empathy as well as a measure on prosodic production ability.

Conclusions

These data indicate, for the first time, that areas that are important for prosody production may also be utilized for prosody perception, as well as other aspects of social communication and social understanding, such as aspects of empathy and prosodic ability.     

Low-Arousal Speech Noise Improves Performance in N-Back Task: An ERP Study

The relationship between noise and human performance is a crucial topic in ergonomic research. However, the brain dynamics of the emotional arousal effects of background noises are still unclear. The current study employed meaningless speech noises in the n-back working memory task to explore the changes of event-related potentials (ERPs) elicited by the noises with low arousal level vs. high arousal level. We found that the memory performance in low arousal condition were improved compared with the silent and the high arousal conditions; participants responded more quickly and had larger P2 and P3 amplitudes in low arousal condition while the performance and ERP components showed no significant difference between high arousal and silent conditions. These findings suggested that the emotional arousal dimension of background noises had a significant influence on human working memory performance, and that this effect was independent of the acoustic characteristics of noises (e.g., intensity) and the meaning of speech materials. The current findings improve our understanding of background noise effects on human performance and lay the ground for the investigation of patients with attention deficits.     

How bodies and voices interact in early emotion perception

Successful social communication draws strongly on the correct interpretation of others' body and vocal expressions. Both can provide emotional information and often occur simultaneously. Yet their interplay has hardly been studied. Using electroencephalography, we investigated the temporal development underlying their neural interaction in auditory and visual perception. In particular, we tested whether this interaction qualifies as true integration following multisensory integration principles such as inverse effectiveness. Emotional vocalizations were embedded in either low or high levels of noise and presented with or without video clips of matching emotional body expressions. In both, high and low noise conditions, a reduction in auditory N100 amplitude was observed for audiovisual stimuli. However, only under high noise, the N100 peaked earlier in the audiovisual than the auditory condition, suggesting facilitatory effects as predicted by the inverse effectiveness principle. Similarly, we observed earlier N100 peaks in response to emotional compared to neutral audiovisual stimuli. This was not the case in the unimodal auditory condition. Furthermore, suppression of beta-band oscillations (15-25 Hz) primarily reflecting biological motion perception was modulated 200-400 ms after the vocalization. While larger differences in suppression between audiovisual and audio stimuli in high compared to low noise levels were found for emotional stimuli, no such difference was observed for neutral stimuli. This observation is in accordance with the inverse effectiveness principle and suggests a modulation of integration by emotional content. Overall, results show that ecologically valid, complex stimuli such as joined body and vocal expressions are effectively integrated very early in processing.     

Hemispheric Asymmetry in Prosody Perception by Healthy Subjects and Schizophrenic Patients

The interhemispheric interactions in perception of Russian prosody were studied in the norm and in schizophrenia as a clinical model of impaired hemispheric interactions. Monaural presentation of stimuli and binaural presentation in a free acoustical field were used. Sentences with main variants of Russian prosodic intonations were used as stimuli. The response time and the number of erroneous responses were recorded. In binaural listening without headphones, no significant difference in the percent of errors in identifying the emotional prosody was found between healthy subjects and schizophrenics. Compared with the healthy subjects, the patients made more errors in understanding the logical stress and fewer errors in understanding the syntagmatic segmentation. By response time, a significant dominance of the left ear was revealed in the healthy subjects during monaural listening to sentences with emotional prosody and complete or incomplete sentences, whereas no significant ear dominance was found in the schizophrenics. During monaural listening to sentences with logical stress, the response time was shorter when stimuli were presented to the right ear both in the healthy subjects and in the schizophrenics. The results testified that the functional brain asymmetry in schizophrenics is flattened. The flattening was less evident in the perception of a logical stress in a sentence and did not significantly affect the efficiency of identification of emotional prosody and syntagmatic segmentation of a sentence.     

Ontogenetic features of psychophysiological mechanisms of perception of speech emotional component in musically gifted children

Cerebral mechanisms of musical abilities were explored in musically gifted children. For this purpose, psychophysiological characteristics of perception of emotional speech information were experimentally studied in samples of gifted and ordinary children. Forty six schoolchildren and forty eight musicians of three age groups (7-10, 11-13 and 14-17 years old) participated in the study. In experimental session, a test sentence was presented to a subject through headphones with two emotional intonations (joy and anger) and without emotional expression. A subject had to recognize the type of emotion. His/her answers were recorded. The analysis of variance revealed age- and gender-related features of emotional recognition: boys musicians led the schoolchildren of the same age by 4-6 years in the development of mechanisms of emotional recognition, whereas girls musicians were 1-3 years ahead. Musical education in girls induced the shift of predominant activities for emotional perception in the left hemisphere; in boys, on the contrary, initial distinct dominance of the left hemisphere was not retained in the process of further education.     

Auditory affective norms for German: testing the influence of depression and anxiety on valence and arousal ratings

Background

The study of emotional speech perception and emotional prosody necessitates stimuli with reliable affective norms. However, ratings may be affected by the participants'' current emotional state as increased anxiety and depression have been shown to yield altered neural responding to emotional stimuli. Therefore, the present study had two aims, first to provide a database of emotional speech stimuli and second to probe the influence of depression and anxiety on the affective ratings.

Methodology/Principal Findings

We selected 120 words from the Leipzig Affective Norms for German database (LANG), which includes visual ratings of positive, negative, and neutral word stimuli. These words were spoken by a male and a female native speaker of German with the respective emotional prosody, creating a total set of 240 auditory emotional stimuli. The recordings were rated again by an independent sample of subjects for valence and arousal, yielding groups of highly arousing negative or positive stimuli and neutral stimuli low in arousal. These ratings were correlated with participants'' emotional state measured with the Depression Anxiety Stress Scales (DASS). Higher depression scores were related to more negative valence of negative and positive, but not neutral words. Anxiety scores correlated with increased arousal and more negative valence of negative words.

Conclusions/Significance

These results underscore the importance of representatively distributed depression and anxiety scores in participants of affective rating studies. The LANG-audition database, which provides well-controlled, short-duration auditory word stimuli for the experimental investigation of emotional speech is available in Supporting Information S1.     

Interaction of music and emotional stimuli in modulating working memory in macaque monkeys

Background music is one of the most frequently encountered contextual factors that affect cognitive and emotional functions in humans. However, it is still unclear whether music induces similar effects in nonhuman primates. Answering this question might bring insight to the long‐lasting question regarding the ability of nonhuman primates in perceiving and dissociating music from other nonmusical acoustic information. In the present study, macaque monkeys were trained to perform a working memory task that required matching visual stimuli. These stimuli had different emotional content (neutral, negative, and positive). Monkeys performed the task within different background acoustic conditions (music, same‐intensity noise, and silence). We hypothesized that the auditory stimuli might interact with emotional information of visual stimuli and modulate monkeys’ performance. Furthermore, if the effects of music and noise on monkeys’ behavioral measures differ it would mean that monkeys perceived and processed music differently. We found that, monkeys committed more errors and were slower when they encountered stimuli with negative or positive emotional content. In the presence of music, the influence of emotional stimuli on monkeys’ performance significantly differed from those of the neutral stimuli, however, in the presence of noise, the effects of emotional stimuli on monkeys’ performance were not distinguishable. The dissociable effects of music and noise on monkeys’ performance show that the effects of emotional stimuli were dependent on the background acoustic conditions. Our findings indicate that background music and the same‐intensity noise were differentially perceived by monkeys and influenced their cognitive functions.     

Ontogeny-specific interaction of psychophysiological mechanisms of emotional perception and educational achievement in students

In order to explore the process of adaptation of children to school environment psychophysiological characteristics of perception of emotional speech information and school progress were experimentally studied. Forty-six schoolchildren of three age groups (7-10, 11-13, and 14-17 years old) participated in the study. In experimental session, a test sentence was presented to a subject through headphones with two emotional intonations (joy and anger) and without emotional expression. A subject had to recognize the type of emotion. His/her answers were recorded. School progress was determined by year grades in Russian, foreign language, and mathematics. Analysis of variance and linear regression analysis showed that ontogenetic features of a correlation between psychophysiological mechanisms of emotion recognition and school progress were gender- and subject-dependent. This correlation was stronger in 7-13-year-old children than in senior children. This age boundary was passed by the girls earlier than by the boys.     

Speech Perception and Localisation with SCORE Bimodal: A Loudness Normalisation Strategy for Combined Cochlear Implant and Hearing Aid Stimulation

A significant fraction of newly implanted cochlear implant recipients use a hearing aid in their non-implanted ear. SCORE bimodal is a sound processing strategy developed for this configuration, aimed at normalising loudness perception and improving binaural loudness balance. Speech perception performance in quiet and noise and sound localisation ability of six bimodal listeners were measured with and without application of SCORE. Speech perception in quiet was measured either with only acoustic, only electric, or bimodal stimulation, at soft and normal conversational levels. For speech in quiet there was a significant improvement with application of SCORE. Speech perception in noise was measured for either steady-state noise, fluctuating noise, or a competing talker, at conversational levels with bimodal stimulation. For speech in noise there was no significant effect of application of SCORE. Modelling of interaural loudness differences in a long-term-average-speech-spectrum-weighted click train indicated that left-right discrimination of sound sources can improve with application of SCORE. As SCORE was found to leave speech perception unaffected or to improve it, it seems suitable for implementation in clinical devices.     

Perceptual learning in children during perception of the emotional component of speech utterances: The influence of age and gender

Perceptual learning was examined during perception of the emotional component of speech when a valid signal was presented against a background of noise to 7-to 17-year-old boys and girls. Accuracy of emotion recognition (AR) and response time (RT) were recorded in two consecutive test series. Analysis of variance of the results demonstrated a high significance of the series order for both the RT and the AR. The study revealed the ontogenetic features of the characteristics of perceptual learning under these conditions. The most notable changes were observed in the time of discriminating emotions. Mathematically, perceptual learning was modeled exponentially. Model parameters such as the rate of perceptual learning and the initial level of perceptual performance were estimated. It was found that the rate of learning, calculated for both the AR and the RT, had close values and similar trends of changes in both the male and the female samples. This suggests that the rate of perceptual learning is an invariant characteristic of perception of the emotional component of speech in a child. The initial level of perceptual performance increased as a function of age, more notably in the boys than in the girls. The initial level of the RT decreased with age in the girls and almost did not depend on age in the boys. However, the integral AR suggests that the age-related changes in the initial level of the integral perception of emotions are identical in girls and boys.     

Listening to Puns Elicits the Co-Activation of Alternative Homophone Meanings during Language Production

Recent evidence suggests that lexical-semantic activation spread during language production can be dynamically shaped by contextual factors. In this study we investigated whether semantic processing modes can also affect lexical-semantic activation during word production. Specifically, we tested whether the processing of linguistic ambiguities, presented in the form of puns, has an influence on the co-activation of unrelated meanings of homophones in a subsequent language production task. In a picture-word interference paradigm with word distractors that were semantically related or unrelated to the non-depicted meanings of homophones we found facilitation induced by related words only when participants listened to puns before object naming, but not when they heard jokes with unambiguous linguistic stimuli. This finding suggests that a semantic processing mode of ambiguity perception can induce the co-activation of alternative homophone meanings during speech planning.     

Auditory Perception of Voice Signals with Abnormal Sounding in Children

Invariant and noise-proof speech understanding is an important human ability, ensured by several mechanisms of the audioverbal system, which develops parallel to mastering linguistic rules. It is a fundamental problem of speech studies to clarify the mechanisms of this understanding, especially their role in the speech development. The article deals with of the regularities of auditory word recognition in noise by preschool children (healthy and with speech development disorders) and patients with cochlear implants. The authors studied the recognition of words using pictures (by children) and verbal monitoring, when the subjects were stimulated by isolated words with one or all syllables in noise. The study showed that children's ability to perceive distorted words develops in ontogeny and is closely related to the development of mental processes and mastering linguistic rules. The data on patients with cochlear implants also confirmed the key role of the central factors in understanding distorted speech.     

Lip-Reading Aids Word Recognition Most in Moderate Noise: A Bayesian Explanation Using High-Dimensional Feature Space

Watching a speaker''s facial movements can dramatically enhance our ability to comprehend words, especially in noisy environments. From a general doctrine of combining information from different sensory modalities (the principle of inverse effectiveness), one would expect that the visual signals would be most effective at the highest levels of auditory noise. In contrast, we find, in accord with a recent paper, that visual information improves performance more at intermediate levels of auditory noise than at the highest levels, and we show that a novel visual stimulus containing only temporal information does the same. We present a Bayesian model of optimal cue integration that can explain these conflicts. In this model, words are regarded as points in a multidimensional space and word recognition is a probabilistic inference process. When the dimensionality of the feature space is low, the Bayesian model predicts inverse effectiveness; when the dimensionality is high, the enhancement is maximal at intermediate auditory noise levels. When the auditory and visual stimuli differ slightly in high noise, the model makes a counterintuitive prediction: as sound quality increases, the proportion of reported words corresponding to the visual stimulus should first increase and then decrease. We confirm this prediction in a behavioral experiment. We conclude that auditory-visual speech perception obeys the same notion of optimality previously observed only for simple multisensory stimuli.     

Involvement of Right STS in Audio-Visual Integration for Affective Speech Demonstrated Using MEG

Speech and emotion perception are dynamic processes in which it may be optimal to integrate synchronous signals emitted from different sources. Studies of audio-visual (AV) perception of neutrally expressed speech demonstrate supra-additive (i.e., where AV>[unimodal auditory+unimodal visual]) responses in left STS to crossmodal speech stimuli. However, emotions are often conveyed simultaneously with speech; through the voice in the form of speech prosody and through the face in the form of facial expression. Previous studies of AV nonverbal emotion integration showed a role for right (rather than left) STS. The current study therefore examined whether the integration of facial and prosodic signals of emotional speech is associated with supra-additive responses in left (cf. results for speech integration) or right (due to emotional content) STS. As emotional displays are sometimes difficult to interpret, we also examined whether supra-additive responses were affected by emotional incongruence (i.e., ambiguity). Using magnetoencephalography, we continuously recorded eighteen participants as they viewed and heard AV congruent emotional and AV incongruent emotional speech stimuli. Significant supra-additive responses were observed in right STS within the first 250 ms for emotionally incongruent and emotionally congruent AV speech stimuli, which further underscores the role of right STS in processing crossmodal emotive signals.     

Speech Recognition in Natural Background Noise

In the real world, human speech recognition nearly always involves listening in background noise. The impact of such noise on speech signals and on intelligibility performance increases with the separation of the listener from the speaker. The present behavioral experiment provides an overview of the effects of such acoustic disturbances on speech perception in conditions approaching ecologically valid contexts. We analysed the intelligibility loss in spoken word lists with increasing listener-to-speaker distance in a typical low-level natural background noise. The noise was combined with the simple spherical amplitude attenuation due to distance, basically changing the signal-to-noise ratio (SNR). Therefore, our study draws attention to some of the most basic environmental constraints that have pervaded spoken communication throughout human history. We evaluated the ability of native French participants to recognize French monosyllabic words (spoken at 65.3 dB(A), reference at 1 meter) at distances between 11 to 33 meters, which corresponded to the SNRs most revealing of the progressive effect of the selected natural noise (−8.8 dB to −18.4 dB). Our results showed that in such conditions, identity of vowels is mostly preserved, with the striking peculiarity of the absence of confusion in vowels. The results also confirmed the functional role of consonants during lexical identification. The extensive analysis of recognition scores, confusion patterns and associated acoustic cues revealed that sonorant, sibilant and burst properties were the most important parameters influencing phoneme recognition. . Altogether these analyses allowed us to extract a resistance scale from consonant recognition scores. We also identified specific perceptual consonant confusion groups depending of the place in the words (onset vs. coda). Finally our data suggested that listeners may access some acoustic cues of the CV transition, opening interesting perspectives for future studies.     

Changes in the EEG spectral power during perception of neutral and emotionally salient words in schizophrenic patients, their relatives and healthy individuals from the general population

To search for EEG-correlates of emotional processing that might be indicators of genetic predisposition to schizophrenia, changes in EEG spectral power during perception of neutral and emotionally salient words were examined in 36 schizophrenic patients, 50 of their unaffected first-degree relatives, and 47 healthy individuals without any family history of psychoses. In healthy persons, passive listening to neutral words induced minimum changes in cortical rhythmical activity, predominantly in the form of synchronization of slow and fast waves, whereas perception of emotional words was followed by a generalized depression of the alpha and beta1 activity and a locally specific decrease in the power of theta and beta2 frequency bands. The patients and their relatives showed a decrease in the alpha and beta1 activity simultaneously with an increase in the power of delta activity in response to both groups of words. Thus, in the patients and their relatives, reactions to neutral and emotional words were ulterior as a result of augmented reactions to the neutral words. These findings suggest that the EEG changes reflect familial and possibly hereditable abnormal involuntary attention. No prominent decrease in reactivity to emotional stimuli was revealed in schizophrenic families.