Speech Recognition in Natural Background Noise

In the real world, human speech recognition nearly always involves listening in background noise. The impact of such noise on speech signals and on intelligibility performance increases with the separation of the listener from the speaker. The present behavioral experiment provides an overview of the effects of such acoustic disturbances on speech perception in conditions approaching ecologically valid contexts. We analysed the intelligibility loss in spoken word lists with increasing listener-to-speaker distance in a typical low-level natural background noise. The noise was combined with the simple spherical amplitude attenuation due to distance, basically changing the signal-to-noise ratio (SNR). Therefore, our study draws attention to some of the most basic environmental constraints that have pervaded spoken communication throughout human history. We evaluated the ability of native French participants to recognize French monosyllabic words (spoken at 65.3 dB(A), reference at 1 meter) at distances between 11 to 33 meters, which corresponded to the SNRs most revealing of the progressive effect of the selected natural noise (−8.8 dB to −18.4 dB). Our results showed that in such conditions, identity of vowels is mostly preserved, with the striking peculiarity of the absence of confusion in vowels. The results also confirmed the functional role of consonants during lexical identification. The extensive analysis of recognition scores, confusion patterns and associated acoustic cues revealed that sonorant, sibilant and burst properties were the most important parameters influencing phoneme recognition. . Altogether these analyses allowed us to extract a resistance scale from consonant recognition scores. We also identified specific perceptual consonant confusion groups depending of the place in the words (onset vs. coda). Finally our data suggested that listeners may access some acoustic cues of the CV transition, opening interesting perspectives for future studies.     

Visual Word Recognition in Deaf Readers: Lexicality Is Modulated by Communication Mode

Evidence indicates that adequate phonological abilities are necessary to develop proficient reading skills and that later in life phonology also has a role in the covert visual word recognition of expert readers. Impairments of acoustic perception, such as deafness, can lead to atypical phonological representations of written words and letters, which in turn can affect reading proficiency. Here, we report an experiment in which young adults with different levels of acoustic perception (i.e., hearing and deaf individuals) and different modes of communication (i.e., hearing individuals using spoken language, deaf individuals with a preference for sign language, and deaf individuals using the oral modality with less or no competence in sign language) performed a visual lexical decision task, which consisted of categorizing real words and consonant strings. The lexicality effect was restricted to deaf signers who responded faster to real words than consonant strings, showing over-reliance on whole word lexical processing of stimuli. No effect of stimulus type was found in deaf individuals using the oral modality or in hearing individuals. Thus, mode of communication modulates the lexicality effect. This suggests that learning a sign language during development shapes visuo-motor representations of words, which are tuned to the actions used to express them (phono-articulatory movements vs. hand movements) and to associated perceptions. As these visuo-motor representations are elicited during on-line linguistic processing and can overlap with the perceptual-motor processes required to execute the task, they can potentially produce interference or facilitation effects.     

Effects of Early Bilingual Experience with a Tone and a Non-Tone Language on Speech-Music Integration

We investigated music and language processing in a group of early bilinguals who spoke a tone language and a non-tone language (Cantonese and Dutch). We assessed online speech-music processing interactions, that is, interactions that occur when speech and music are processed simultaneously in songs, with a speeded classification task. In this task, participants judged sung pseudowords either musically (based on the direction of the musical interval) or phonologically (based on the identity of the sung vowel). We also assessed longer-term effects of linguistic experience on musical ability, that is, the influence of extensive prior experience with language when processing music. These effects were assessed with a task in which participants had to learn to identify musical intervals and with four pitch-perception tasks. Our hypothesis was that due to their experience in two different languages using lexical versus intonational tone, the early Cantonese-Dutch bilinguals would outperform the Dutch control participants. In online processing, the Cantonese-Dutch bilinguals processed speech and music more holistically than controls. This effect seems to be driven by experience with a tone language, in which integration of segmental and pitch information is fundamental. Regarding longer-term effects of linguistic experience, we found no evidence for a bilingual advantage in either the music-interval learning task or the pitch-perception tasks. Together, these results suggest that being a Cantonese-Dutch bilingual does not have any measurable longer-term effects on pitch and music processing, but does have consequences for how speech and music are processed jointly.     

The Effect of Automatic Gain Control Structure and Release Time on Cochlear Implant Speech Intelligibility

Nucleus cochlear implant systems incorporate a fast-acting front-end automatic gain control (AGC), sometimes called a compression limiter. The objective of the present study was to determine the effect of replacing the front-end compression limiter with a newly proposed envelope profile limiter. A secondary objective was to investigate the effect of AGC speed on cochlear implant speech intelligibility. The envelope profile limiter was located after the filter bank and reduced the gain when the largest of the filter bank envelopes exceeded the compression threshold. The compression threshold was set equal to the saturation level of the loudness growth function (i.e. the envelope level that mapped to the maximum comfortable current level), ensuring that no envelope clipping occurred. To preserve the spectral profile, the same gain was applied to all channels. Experiment 1 compared sentence recognition with the front-end limiter and with the envelope profile limiter, each with two release times (75 and 625 ms). Six implant recipients were tested in quiet and in four-talker babble noise, at a high presentation level of 89 dB SPL. Overall, release time had a larger effect than the AGC type. With both AGC types, speech intelligibility was lower for the 75 ms release time than for the 625 ms release time. With the shorter release time, the envelope profile limiter provided higher group mean scores than the front-end limiter in quiet, but there was no significant difference in noise. Experiment 2 measured sentence recognition in noise as a function of presentation level, from 55 to 89 dB SPL. The envelope profile limiter with 625 ms release time yielded better scores than the front-end limiter with 75 ms release time. A take-home study showed no clear pattern of preferences. It is concluded that the envelope profile limiter is a feasible alternative to a front-end compression limiter.     

Solutions to the cocktail party problem in insects: selective filters, spatial release from masking and gain control in tropical crickets

Background

Insects often communicate by sound in mixed species choruses; like humans and many vertebrates in crowded social environments they thus have to solve cocktail-party-like problems in order to ensure successful communication with conspecifics. This is even more a problem in species-rich environments like tropical rainforests, where background noise levels of up to 60 dB SPL have been measured.

Principal Findings

Using neurophysiological methods we investigated the effect of natural background noise (masker) on signal detection thresholds in two tropical cricket species Paroecanthus podagrosus and Diatrypa sp., both in the laboratory and outdoors. We identified three ‘bottom-up’ mechanisms which contribute to an excellent neuronal representation of conspecific signals despite the masking background. First, the sharply tuned frequency selectivity of the receiver reduces the amount of masking energy around the species-specific calling song frequency. Laboratory experiments yielded an average signal-to-noise ratio (SNR) of −8 dB, when masker and signal were broadcast from the same side. Secondly, displacing the masker by 180° from the signal improved SNRs by further 6 to 9 dB, a phenomenon known as spatial release from masking. Surprisingly, experiments carried out directly in the nocturnal rainforest yielded SNRs of about −23 dB compared with those in the laboratory with the same masker, where SNRs reached only −14.5 and −16 dB in both species. Finally, a neuronal gain control mechanism enhances the contrast between the responses to signals and the masker, by inhibition of neuronal activity in interstimulus intervals.

Conclusions

Thus, conventional speaker playbacks in the lab apparently do not properly reconstruct the masking noise situation in a spatially realistic manner, since under real world conditions multiple sound sources are spatially distributed in space. Our results also indicate that without knowledge of the receiver properties and the spatial release mechanisms the detrimental effect of noise may be strongly overestimated.     

Concurrent word generation and motor performance: further evidence for language-motor interaction

Embodied/modality-specific theories of semantic memory propose that sensorimotor representations play an important role in perception and action. A large body of evidence supports the notion that concepts involving human motor action (i.e., semantic-motor representations) are processed in both language and motor regions of the brain. However, most studies have focused on perceptual tasks, leaving unanswered questions about language-motor interaction during production tasks. Thus, we investigated the effects of shared semantic-motor representations on concurrent language and motor production tasks in healthy young adults, manipulating the semantic task (motor-related vs. nonmotor-related words) and the motor task (i.e., standing still and finger-tapping). In Experiment 1 (n = 20), we demonstrated that motor-related word generation was sufficient to affect postural control. In Experiment 2 (n = 40), we demonstrated that motor-related word generation was sufficient to facilitate word generation and finger tapping. We conclude that engaging semantic-motor representations can have a reciprocal influence on motor and language production. Our study provides additional support for functional language-motor interaction, as well as embodied/modality-specific theories.     

Language Differences in the Brain Network for Reading in Naturalistic Story Reading and Lexical Decision

Differences in how writing systems represent language raise important questions about whether there could be a universal functional architecture for reading across languages. In order to study potential language differences in the neural networks that support reading skill, we collected fMRI data from readers of alphabetic (English) and morpho-syllabic (Chinese) writing systems during two reading tasks. In one, participants read short stories under conditions that approximate natural reading, and in the other, participants decided whether individual stimuli were real words or not. Prior work comparing these two writing systems has overwhelmingly used meta-linguistic tasks, generally supporting the conclusion that the reading system is organized differently for skilled readers of Chinese and English. We observed that language differences in the reading network were greatly dependent on task. In lexical decision, a pattern consistent with prior research was observed in which the Middle Frontal Gyrus (MFG) and right Fusiform Gyrus (rFFG) were more active for Chinese than for English, whereas the posterior temporal sulcus was more active for English than for Chinese. We found a very different pattern of language effects in a naturalistic reading paradigm, during which significant differences were only observed in visual regions not typically considered specific to the reading network, and the middle temporal gyrus, which is thought to be important for direct mapping of orthography to semantics. Indeed, in areas that are often discussed as supporting distinct cognitive or linguistic functions between the two languages, we observed interaction. Specifically, language differences were most pronounced in MFG and rFFG during the lexical decision task, whereas no language differences were observed in these areas during silent reading of text for comprehension.     

Nonnative Audiovisual Speech Perception in Noise: Dissociable Effects of the Speaker and Listener

Nonnative speech poses a challenge to speech perception, especially in challenging listening environments. Audiovisual (AV) cues are known to improve native speech perception in noise. The extent to which AV cues benefit nonnative speech perception in noise, however, is much less well-understood. Here, we examined native American English-speaking and native Korean-speaking listeners'' perception of English sentences produced by a native American English speaker and a native Korean speaker across a range of signal-to-noise ratios (SNRs;−4 to −20 dB) in audio-only and audiovisual conditions. We employed psychometric function analyses to characterize the pattern of AV benefit across SNRs. For native English speech, the largest AV benefit occurred at intermediate SNR (i.e. −12 dB); but for nonnative English speech, the largest AV benefit occurred at a higher SNR (−4 dB). The psychometric function analyses demonstrated that the AV benefit patterns were different between native and nonnative English speech. The nativeness of the listener exerted negligible effects on the AV benefit across SNRs. However, the nonnative listeners'' ability to gain AV benefit in native English speech was related to their proficiency in English. These findings suggest that the native language background of both the speaker and listener clearly modulate the optimal use of AV cues in speech recognition.     

Factors Limiting Performance in a Multitone Intensity-Discrimination Task: Disentangling Non-Optimal Decision Weights and Increased Internal Noise

To identify factors limiting performance in multitone intensity discrimination, we presented sequences of five pure tones alternating in level between loud (85 dB SPL) and soft (30, 55, or 80 dB SPL). In the “overall-intensity task”, listeners detected a level increment on all of the five tones. In the “masking task”, the level increment was imposed only on the soft tones, rendering the soft tones targets and loud tones task-irrelevant maskers. Decision weights quantifying the importance of the five tone levels for the decision were estimated using methods of molecular psychophysics. Compatible with previous studies, listeners placed higher weights on the loud tones than on the soft tones in the overall-intensity condition. In the masking task, the decisions were systematically influenced by the to-be-ignored loud tones (maskers). Using a maximum-likelihood technique, we estimated the internal noise variance and tested whether the internal noise was higher in the alternating-level five-tone sequences than in sequences presenting only the soft or only the loud tones. For the overall-intensity task, we found no evidence for increased internal noise, but listeners applied suboptimal decision weights. These results are compatible with the hypothesis that the presence of the loud tones does not impair the precision of the representation of the intensity of the soft tones available at the decision stage, but that this information is not used in an optimal fashion due to a difficulty in attending to the soft tones. For the masking task, in some cases our data indicated an increase in internal noise. Additionally, listeners applied suboptimal decision weights. The maximum-likelihood analyses we developed should also be useful for other tasks or other sensory modalities.     

Large Scale Brain Functional Networks Support Sentence Comprehension: Evidence from Both Explicit and Implicit Language Tasks

Previous studies have indicated that sentences are comprehended via widespread brain regions in the fronto-temporo-parietal network in explicit language tasks (e.g., semantic congruency judgment tasks), and through restricted temporal or frontal regions in implicit language tasks (e.g., font size judgment tasks). This discrepancy has raised questions regarding a common network for sentence comprehension that acts regardless of task effect and whether different tasks modulate network properties. To this end, we constructed brain functional networks based on 27 subjects’ fMRI data that was collected while performing explicit and implicit language tasks. We found that network properties and network hubs corresponding to the implicit language task were similar to those associated with the explicit language task. We also found common hubs in occipital, temporal and frontal regions in both tasks. Compared with the implicit language task, the explicit language task resulted in greater global efficiency and increased integrated betweenness centrality of the left inferior frontal gyrus, which is a key region related to sentence comprehension. These results suggest that brain functional networks support both explicit and implicit sentence comprehension; in addition, these two types of language tasks may modulate the properties of brain functional networks.     

Role of joint language control during cross-language communication: evidence from cross-frequency coupling

How do bilingual interlocutors inhibit interference from the non-target language to achieve brain-to-brain information exchange in a task to simulate a bilingual speaker–listener interaction. In the current study, two electroencephalogram devices were employed to record pairs of participants’ performances in a joint language switching task. Twenty-eight (14 pairs) unbalanced Chinese–English bilinguals (L1 Chinese) were instructed to name pictures in the appropriate language according to the cue. The phase-amplitude coupling analysis was employed to reveal the large-scale brain network responsible for joint language control between interlocutors. We found that (1) speakers and listeners coordinately suppressed cross-language interference through cross-frequency coupling, as shown in the increased delta/theta phase-amplitude and delta/alpha phase-amplitude coupling when switching to L2 than switching to L1; (2) speakers and listeners were both able to simultaneously inhibit cross-person item-level interference which was demonstrated by stronger cross-frequency coupling in the cross person condition compared to the within person condition. These results indicate that current bilingual models (e.g., the inhibitory control model) should incorporate mechanisms that address inhibiting interference sourced in both language and person (i.e., cross-language and cross-person item-level interference) synchronously through joint language control in dynamic cross-language communication.     

Paradoxical relationship between speed and accuracy in olfactory figure-background segregation

In natural settings, many stimuli impinge on our sensory organs simultaneously. Parsing these sensory stimuli into perceptual objects is a fundamental task faced by all sensory systems. Similar to other sensory modalities, increased odor backgrounds decrease the detectability of target odors by the olfactory system. The mechanisms by which background odors interfere with the detection and identification of target odors are unknown. Here we utilized the framework of the Drift Diffusion Model (DDM) to consider possible interference mechanisms in an odor detection task. We first considered pure effects of background odors on either signal or noise in the decision-making dynamics and showed that these produce different predictions about decision accuracy and speed. To test these predictions, we trained mice to detect target odors that are embedded in random background mixtures in a two-alternative choice task. In this task, the inter-trial interval was independent of behavioral reaction times to avoid motivating rapid responses. We found that increased backgrounds reduce mouse performance but paradoxically also decrease reaction times, suggesting that noise in the decision making process is increased by backgrounds. We further assessed the contributions of background effects on both noise and signal by fitting the DDM to the behavioral data. The models showed that background odors affect both the signal and the noise, but that the paradoxical relationship between trial difficulty and reaction time is caused by the added noise.     

Reading “Sun” and Looking Up: The Influence of Language on Saccadic Eye Movements in the Vertical Dimension

Traditionally, language processing has been attributed to a separate system in the brain, which supposedly works in an abstract propositional manner. However, there is increasing evidence suggesting that language processing is strongly interrelated with sensorimotor processing. Evidence for such an interrelation is typically drawn from interactions between language and perception or action. In the current study, the effect of words that refer to entities in the world with a typical location (e.g., sun, worm) on the planning of saccadic eye movements was investigated. Participants had to perform a lexical decision task on visually presented words and non-words. They responded by moving their eyes to a target in an upper (lower) screen position for a word (non-word) or vice versa. Eye movements were faster to locations compatible with the word''s referent in the real world. These results provide evidence for the importance of linguistic stimuli in directing eye movements, even if the words do not directly transfer directional information.     

Linguistic feature analysis for protein interaction extraction

Background  

The rapid growth of the amount of publicly available reports on biomedical experimental results has recently caused a boost of text mining approaches for protein interaction extraction. Most approaches rely implicitly or explicitly on linguistic, i.e., lexical and syntactic, data extracted from text. However, only few attempts have been made to evaluate the contribution of the different feature types. In this work, we contribute to this evaluation by studying the relative importance of deep syntactic features, i.e., grammatical relations, shallow syntactic features (part-of-speech information) and lexical features. For this purpose, we use a recently proposed approach that uses support vector machines with structured kernels.     

Acoustic signal perception in a noisy habitat: lessons from synchronising insects

Acoustically communicating animals often have to cope with ambient noise that has the potential to interfere with the perception of conspecific signals. Here we use the synchronous display of mating signals in males of the tropical katydid Mecopoda elongata in order to assess the influence of nocturnal rainforest noise on signal perception. Loud background noise may disturb chorus synchrony either by masking the signals of males or by interaction of noisy events with the song oscillator. Phase-locked synchrony of males was studied under various signal-to-noise ratios (SNRs) using either native noise or the audio component of noise (<9 kHz). Synchronous entrainment was lost at a SNR of -3 dB when native noise was used, whereas with the audio component still 50% of chirp periods matched the pacer period at a SNR of -7 dB. Since the chirp period of solo singing males remained almost unaffected by noise, our results suggest that masking interference limits chorus synchrony by rendering conspecific signals ambiguous. Further, entrainment with periodic artificial signals indicates that synchrony is achieved by ignoring heterospecific signals and attending to a conspecific signal period. Additionally, the encoding of conspecific chirps was studied in an auditory neuron under the same background noise regimes.     

Effects of noise bandwidth and amplitude modulation on masking in frog auditory midbrain neurons

Natural auditory scenes such as frog choruses consist of multiple sound sources (i.e., individual vocalizing males) producing sounds that overlap extensively in time and spectrum, often in the presence of other biotic and abiotic background noise. Detection of a signal in such environments is challenging, but it is facilitated when the noise shares common amplitude modulations across a wide frequency range, due to a phenomenon called comodulation masking release (CMR). Here, we examined how properties of the background noise, such as its bandwidth and amplitude modulation, influence the detection threshold of a target sound (pulsed amplitude modulated tones) by single neurons in the frog auditory midbrain. We found that for both modulated and unmodulated masking noise, masking was generally stronger with increasing bandwidth, but it was weakened for the widest bandwidths. Masking was less for modulated noise than for unmodulated noise for all bandwidths. However, responses were heterogeneous, and only for a subpopulation of neurons the detection of the probe was facilitated when the bandwidth of the modulated masker was increased beyond a certain bandwidth - such neurons might contribute to CMR. We observed evidence that suggests that the dips in the noise amplitude are exploited by TS neurons, and observed strong responses to target signals occurring during such dips. However, the interactions between the probe and masker responses were nonlinear, and other mechanisms, e.g., selective suppression of the response to the noise, may also be involved in the masking release.     

Effect of masking on interhemispheric and bilateral asymmetry of long-latency auditory evoked potentials

Considering the effects of noise on the amplitude of long-latent auditory evoked potentials, it is concluded that masking interference diminishes the right hemisphere dominance in the processing of the non-speech acoustic information. The decrease is observed both with the monotic and dichotic delivery of signals and noise. With monotic presentation masking interference also decreases the extent of the lateralization of monaurally delivered information predominantly to the contralateral hemisphere.     

Influence of Musical Training on Understanding Voiced and Whispered Speech in Noise

This study tested the hypothesis that the previously reported advantage of musicians over non-musicians in understanding speech in noise arises from more efficient or robust coding of periodic voiced speech, particularly in fluctuating backgrounds. Speech intelligibility was measured in listeners with extensive musical training, and in those with very little musical training or experience, using normal (voiced) or whispered (unvoiced) grammatically correct nonsense sentences in noise that was spectrally shaped to match the long-term spectrum of the speech, and was either continuous or gated with a 16-Hz square wave. Performance was also measured in clinical speech-in-noise tests and in pitch discrimination. Musicians exhibited enhanced pitch discrimination, as expected. However, no systematic or statistically significant advantage for musicians over non-musicians was found in understanding either voiced or whispered sentences in either continuous or gated noise. Musicians also showed no statistically significant advantage in the clinical speech-in-noise tests. Overall, the results provide no evidence for a significant difference between young adult musicians and non-musicians in their ability to understand speech in noise.     

诱发性耳声发射的掩蔽

３４例听觉正常受试者（共４８耳）进行疏波短声诱发性耳声发射（ＥＯＡＥ）掩蔽实验,项目包括同侧同时掩蔽、同侧后掩蔽和对倒后掩蔽。同时掩蔽的掩蔽声是稳态白噪声,后掩蔽的掩蔽声是宽带噪声。同侧同时掩蔽强度达３０ｄＢＳＬ时,未观察到对ＥＯＡＥ的掩蔽效应,但对主观听觉感受有掩蔽作用,表明ＥＯＡＥ的客观属性反映听觉行为有其局限性、同侧及对侧后掩蔽出现掩蔽效应时的掩蔽强度分别为３０和５０ｄＢＳＬ,掩蔽阈约分别为５９和６８ｄＢＳＬ。耳蜗的机械特性－非线性或耳蜗内存在的某种功能性的反馈调节系统可能是同侧后掩蔽的作用机理。下行的对侧橄榄耳蜗内侧束对外毛细胞主动收缩的抑制性作用,可有效解释对倒后掩蔽的ＥＯＡＥ变化。     

Neural Congruency Effects in the Multi-Source Interference Task Vanish in Healthy Youth after Controlling for Conditional Differences in Mean RT

According to the conflict monitoring model of cognitive control, reaction time (RT) in distracter interference tasks (e.g., the Stroop task) is a more precise index of response conflict than stimulus congruency (incongruent vs. congruent). The model therefore predicts that RT should be a reliable predictor of activity in regions of the posterior medial frontal cortex (pMFC) that are posited to detect response conflict. In particular, pMFC activity should be (a) greater in slow-RT than in fast-RT trials within a given task condition (e.g., congruent) and (b) equivalent in RT-matched trials from different conditions (i.e., congruent and incongruent trials). Both of these effects have been observed in functional magnetic resonance imaging (MRI) studies of adults. However, neither effect was observed in a recent study of healthy youth, suggesting that (a) the model does not accurately describe the relationship between RT and pMFC activity in this population or (b) the recent study was characterized by high variability due to a relatively small sample size. To distinguish between these possibilities, we asked a relatively large group of healthy youth (n = 28) to perform a distracter interference task - the multi-source interference task (MSIT) - while we recorded their brain activity with functional MRI. In this relatively large sample, both of the model’s predictions were confirmed. We conclude that the model accurately describes the relationship between pMFC activity and RT in healthy youth, but that additional research is needed to determine whether processes unrelated to response conflict contribute to this relationship.