The effect of visual cues on difficulty ratings for segregation of musical streams in listeners with impaired hearing

Background

Enjoyment of music is an important part of life that may be degraded for people with hearing impairments, especially those using cochlear implants. The ability to follow separate lines of melody is an important factor in music appreciation. This ability relies on effective auditory streaming, which is much reduced in people with hearing impairment, contributing to difficulties in music appreciation. The aim of this study was to assess whether visual cues could reduce the subjective difficulty of segregating a melody from interleaved background notes in normally hearing listeners, those using hearing aids, and those using cochlear implants.

Methodology/Principal Findings

Normally hearing listeners (N = 20), hearing aid users (N = 10), and cochlear implant users (N = 11) were asked to rate the difficulty of segregating a repeating four-note melody from random interleaved distracter notes. The pitch of the background notes was gradually increased or decreased throughout blocks, providing a range of difficulty from easy (with a large pitch separation between melody and distracter) to impossible (with the melody and distracter completely overlapping). Visual cues were provided on half the blocks, and difficulty ratings for blocks with and without visual cues were compared between groups. Visual cues reduced the subjective difficulty of extracting the melody from the distracter notes for normally hearing listeners and cochlear implant users, but not hearing aid users.

Conclusion/Significance

Simple visual cues may improve the ability of cochlear implant users to segregate lines of music, thus potentially increasing their enjoyment of music. More research is needed to determine what type of acoustic cues to encode visually in order to optimise the benefits they may provide.     

Perception of Binaural Cues Develops in Children Who Are Deaf through Bilateral Cochlear Implantation

There are significant challenges to restoring binaural hearing to children who have been deaf from an early age. The uncoordinated and poor temporal information available from cochlear implants distorts perception of interaural timing differences normally important for sound localization and listening in noise. Moreover, binaural development can be compromised by bilateral and unilateral auditory deprivation. Here, we studied perception of both interaural level and timing differences in 79 children/adolescents using bilateral cochlear implants and 16 peers with normal hearing. They were asked on which side of their head they heard unilaterally or bilaterally presented click- or electrical pulse- trains. Interaural level cues were identified by most participants including adolescents with long periods of unilateral cochlear implant use and little bilateral implant experience. Interaural timing cues were not detected by new bilateral adolescent users, consistent with previous evidence. Evidence of binaural timing detection was, for the first time, found in children who had much longer implant experience but it was marked by poorer than normal sensitivity and abnormally strong dependence on current level differences between implants. In addition, children with prior unilateral implant use showed a higher proportion of responses to their first implanted sides than children implanted simultaneously. These data indicate that there are functional repercussions of developing binaural hearing through bilateral cochlear implants, particularly when provided sequentially; nonetheless, children have an opportunity to use these devices to hear better in noise and gain spatial hearing.     

Binaural Fusion and Listening Effort in Children Who Use Bilateral Cochlear Implants: A Psychoacoustic and Pupillometric Study

Bilateral cochlear implants aim to provide hearing to both ears for children who are deaf and promote binaural/spatial hearing. Benefits are limited by mismatched devices and unilaterally-driven development which could compromise the normal integration of left and right ear input. We thus asked whether children hear a fused image (ie. 1 vs 2 sounds) from their bilateral implants and if this “binaural fusion” reduces listening effort. Binaural fusion was assessed by asking 25 deaf children with cochlear implants and 24 peers with normal hearing whether they heard one or two sounds when listening to bilaterally presented acoustic click-trains/electric pulses (250 Hz trains of 36 ms presented at 1 Hz). Reaction times and pupillary changes were recorded simultaneously to measure listening effort. Bilaterally implanted children heard one image of bilateral input less frequently than normal hearing peers, particularly when intensity levels on each side were balanced. Binaural fusion declined as brainstem asymmetries increased and age at implantation decreased. Children implanted later had access to acoustic input prior to implantation due to progressive deterioration of hearing. Increases in both pupil diameter and reaction time occurred as perception of binaural fusion decreased. Results indicate that, without binaural level cues, children have difficulty fusing input from their bilateral implants to perceive one sound which costs them increased listening effort. Brainstem asymmetries exacerbate this issue. By contrast, later implantation, reflecting longer access to bilateral acoustic hearing, may have supported development of auditory pathways underlying binaural fusion. Improved integration of bilateral cochlear implant signals for children is required to improve their binaural hearing.     

The Effect of Visual Cues on Auditory Stream Segregation in Musicians and Non-Musicians

Background

The ability to separate two interleaved melodies is an important factor in music appreciation. This ability is greatly reduced in people with hearing impairment, contributing to difficulties in music appreciation. The aim of this study was to assess whether visual cues, musical training or musical context could have an effect on this ability, and potentially improve music appreciation for the hearing impaired.

Methods

Musicians (N = 18) and non-musicians (N = 19) were asked to rate the difficulty of segregating a four-note repeating melody from interleaved random distracter notes. Visual cues were provided on half the blocks, and two musical contexts were tested, with the overlap between melody and distracter notes either gradually increasing or decreasing.

Conclusions

Visual cues, musical training, and musical context all affected the difficulty of extracting the melody from a background of interleaved random distracter notes. Visual cues were effective in reducing the difficulty of segregating the melody from distracter notes, even in individuals with no musical training. These results are consistent with theories that indicate an important role for central (top-down) processes in auditory streaming mechanisms, and suggest that visual cues may help the hearing-impaired enjoy music.     

Neuroprosthetic hearing with auditory brainstem implants.

The auditory brainstem implant (ABI) does provide auditory sensations, recognition of environmental sounds and aid in spoken communication in about 300 patients worldwide. It is no more an investigative device but widely accepted for the treatment of patients who have lost hearing due to bilateral tumors of the hearing nerve who transmits the acoustic information from the cochlea to the brain. Most of the implanted patients are completely deaf when the implant is switched off. In contrast to cochlear implants, only few of the implanted patients achieve open-set speech recognition without the help of visual cues. On average, the ABI improves communicative functions like speech recognition at about 30% when compared to lip-reading only. The task for the next years is to improve the outcome of ABI further by developing new less invasive operative approaches as well as new hardware and software for the ABI device.     

Long-Term Asymmetric Hearing Affects Cochlear Implantation Outcomes Differently in Adults with Pre- and Postlingual Hearing Loss

In many countries, a single cochlear implant is offered as a treatment for a bilateral hearing loss. In cases where there is asymmetry in the amount of sound deprivation between the ears, there is a dilemma in choosing which ear should be implanted. In many clinics, the choice of ear has been guided by an assumption that the reorganisation of the auditory pathways caused by longer duration of deafness in one ear is associated with poorer implantation outcomes for that ear. This assumption, however, is mainly derived from studies of early childhood deafness. This study compared outcomes following implantation of the better or poorer ear in cases of long-term hearing asymmetries. Audiological records of 146 adults with bilateral hearing loss using a single hearing aid were reviewed. The unaided ear had 15 to 72 years of unaided severe to profound hearing loss before unilateral cochlear implantation. 98 received the implant in their long-term sound-deprived ear. A multiple regression analysis was conducted to assess the relative contribution of potential predictors to speech recognition performance after implantation. Duration of bilateral significant hearing loss and the presence of a prelingual hearing loss explained the majority of variance in speech recognition performance following cochlear implantation. For participants with postlingual hearing loss, similar outcomes were obtained by implanting either ear. With prelingual hearing loss, poorer outcomes were obtained when implanting the long-term sound-deprived ear, but the duration of the sound deprivation in the implanted ear did not reliably predict outcomes. Contrary to an apparent clinical consensus, duration of sound deprivation in one ear has limited value in predicting speech recognition outcomes of cochlear implantation in that ear. Outcomes of cochlear implantation are more closely related to the period of time for which the brain is deprived of auditory stimulation from both ears.     

音乐绝对音高行为研究述评

音乐绝对音高(musical absolute pitch,AP)能力是一种敏锐的乐音音高知觉分类和记忆能力,表现为在没有任何其它音高参照的情况下,能够对所听到乐音音高进行准确命名或分类。本文从音乐绝对音高甄别方法、基因遗传、早期音乐训练、语言环境对音乐绝对音高能力形成的影响,以及绝对音高与音乐能力、音乐记忆和空间能力方面的研究进行了介绍,提出研究音乐绝对音高的重要性。最后展望了音乐绝对音高领域未来可能的四个研究方向。     

Cortical plasticity induced by short-term multimodal musical rhythm training

Performing music is a multimodal experience involving the visual, auditory, and somatosensory modalities as well as the motor system. Therefore, musical training is an excellent model to study multimodal brain plasticity. Indeed, we have previously shown that short-term piano practice increase the magnetoencephalographic (MEG) response to melodic material in novice players. Here we investigate the impact of piano training using a rhythmic-focused exercise on responses to rhythmic musical material. Musical training with non musicians was conducted over a period of two weeks. One group (sensorimotor-auditory, SA) learned to play a piano sequence with a distinct musical rhythm, another group (auditory, A) listened to, and evaluated the rhythmic accuracy of the performances of the SA-group. Training-induced cortical plasticity was evaluated using MEG, comparing the mismatch negativity (MMN) in response to occasional rhythmic deviants in a repeating rhythm pattern before and after training. The SA-group showed a significantly greater enlargement of MMN and P2 to deviants after training compared to the A- group. The training-induced increase of the rhythm MMN was bilaterally expressed in contrast to our previous finding where the MMN for deviants in the pitch domain showed a larger right than left increase. The results indicate that when auditory experience is strictly controlled during training, involvement of the sensorimotor system and perhaps increased attentional recources that are needed in producing rhythms lead to more robust plastic changes in the auditory cortex compared to when rhythms are simply attended to in the auditory domain in the absence of motor production.     

单侧人工耳蜗植入对学龄前耳聋儿童听觉言语康复的效果影响因素分析

目的:探讨单侧人工耳蜗植入(cochlear implantation,CI)对学龄前耳聋儿童听觉语言康复的治疗效果以及相关影响因素。方法:将我院自2017年1月至2017年12月行CI治疗的学龄前儿童72例行作为研究对象,通过问卷调查手术患儿的相关资料,对可能影响患儿听觉言语康复效果的因素和听觉行为分级(Categories of auditory performance,CAP)以及言语可懂程度分级(Speech intelligibility rating,SIR)结果进行二分类变量的单因素分析,再进行多分类变量的Logistic回归分析评估患儿的治疗效果和影响康复效果的因素。结果:耳聋患儿CI植入年龄、术前平均残余听力、术前佩戴助听器时间、使用人工耳蜗时间和术后语训时间等因素和CAP增长倍数之间有明显的相关性(P0.05),除了上述因素之外还有术前语训时间等因素与治疗后患儿SIR增长倍数存在相关性(P0.05);CI植入年龄、术前平均残余听力和术前佩戴助听器时间对患儿术后CAP的恢复具有影响(P0.05);CI植入年龄、术前佩戴助听器时间、术前语训时间等因素对患儿SIR恢复产生影响(P0.05)。结论:患儿植入人工耳蜗的年龄、术前平均残余听力、术前佩戴助听器时间和术前言语训练时间是影响学龄前耳聋患儿术后听力言语功能恢复的主要因素。     

Auditory rehabilitation

Auditory rehabilitation depends of the cause and the severity of the hearing loss (or deafness). Hearing losses dues to middle ear pathologies can beneficiate of medical or surgical treatments, by ossicular prostheses, if it is necessary to restore the function of the ossicles chain. In the sensorineural hearing losses, with inner ear pathology, the use of auditory aid is immediately considered. In the cases for which they are insufficient because of severity of the hearing loss or not suitable because of local non-tolerance, it is possible to use middle ear implant or cochlear implant. The indications of the auditory brainstern implants remain at this day limited to the total bilateral hearing losses due to a complete destruction of cochleae and auditory nerves. These therapeutic orientations are selected after a multidisciplinary evaluation of the deaf person, evaluation that allows the characterization of the hearing loss and its repercussion. In all the cases, the restoration of a bilateral hearing has to be done if possible, making an improvement of the speech comprehension, mainly in the noisy situations, as well as the localization of the sound sources.     

Cochlear implants: the view from the brain

The cochlear implant arguably is the most successful neural prosthesis. Studies of the responses of the central auditory system to prosthetic electrical stimulation of the cochlea are revealing the success with which electrical stimulation of a deaf ear can mimic acoustic stimulation of a normal-hearing ear. Understanding of the physiology of central auditory structures can lead to improved restoration of hearing with cochlear implants. In turn, the cochlear implant can be exploited as an experimental tool for examining central hearing mechanisms isolated from the effects of cochlear mechanics and transduction.     

The origins of music in auditory scene analysis and the roles of evolution and culture in musical creation

Whether music was an evolutionary adaptation that conferred survival advantages or a cultural creation has generated much debate. Consistent with an evolutionary hypothesis, music is unique to humans, emerges early in development and is universal across societies. However, the adaptive benefit of music is far from obvious. Music is highly flexible, generative and changes rapidly over time, consistent with a cultural creation hypothesis. In this paper, it is proposed that much of musical pitch and timing structure adapted to preexisting features of auditory processing that evolved for auditory scene analysis (ASA). Thus, music may have emerged initially as a cultural creation made possible by preexisting adaptations for ASA. However, some aspects of music, such as its emotional and social power, may have subsequently proved beneficial for survival and led to adaptations that enhanced musical behaviour. Ontogenetic and phylogenetic evidence is considered in this regard. In particular, enhanced auditory–motor pathways in humans that enable movement entrainment to music and consequent increases in social cohesion, and pathways enabling music to affect reward centres in the brain should be investigated as possible musical adaptations. It is concluded that the origins of music are complex and probably involved exaptation, cultural creation and evolutionary adaptation.     

Assessing Musical Abilities Objectively: Construction and Validation of the Profile of Music Perception Skills

A common approach for determining musical competence is to rely on information about individuals’ extent of musical training, but relying on musicianship status fails to identify musically untrained individuals with musical skill, as well as those who, despite extensive musical training, may not be as skilled. To counteract this limitation, we developed a new test battery (Profile of Music Perception Skills; PROMS) that measures perceptual musical skills across multiple domains: tonal (melody, pitch), qualitative (timbre, tuning), temporal (rhythm, rhythm-to-melody, accent, tempo), and dynamic (loudness). The PROMS has satisfactory psychometric properties for the composite score (internal consistency and test-retest r>.85) and fair to good coefficients for the individual subtests (.56 to.85). Convergent validity was established with the relevant dimensions of Gordon’s Advanced Measures of Music Audiation and Musical Aptitude Profile (melody, rhythm, tempo), the Musical Ear Test (rhythm), and sample instrumental sounds (timbre). Criterion validity was evidenced by consistently sizeable and significant relationships between test performance and external musical proficiency indicators in all three studies (.38 to.62, p<.05 to p<.01). An absence of correlations between test scores and a nonmusical auditory discrimination task supports the battery’s discriminant validity (−.05, ns). The interrelationships among the various subtests could be accounted for by two higher order factors, sequential and sensory music processing. A brief version of the full PROMS is introduced as a time-efficient approximation of the full version of the battery.     

Temporal Feature Perception in Cochlear Implant Users

For the perception of timbre of a musical instrument, the attack time is known to hold crucial information. The first 50 to 150 ms of sound onset reflect the excitation mechanism, which generates the sound. Since auditory processing and music perception in particular are known to be hampered in cochlear implant (CI) users, we conducted an electroencephalography (EEG) study with an oddball paradigm to evaluate the processing of small differences in musical sound onset. The first 60 ms of a cornet sound were manipulated in order to examine whether these differences are detected by CI users and normal-hearing controls (NH controls), as revealed by auditory evoked potentials (AEPs). Our analysis focused on the N1 as an exogenous component known to reflect physical stimuli properties as well as on the P2 and the Mismatch Negativity (MMN). Our results revealed different N1 latencies as well as P2 amplitudes and latencies for the onset manipulations in both groups. An MMN could be elicited only in the NH control group. Together with additional findings that suggest an impact of musical training on CI users’ AEPs, our findings support the view that impaired timbre perception in CI users is at partly due to altered sound onset feature detection.     

Auditory hair cell explant co-cultures promote the differentiation of stem cells into bipolar neurons

Auditory neurons, the target neurons of the cochlear implant, degenerate following a sensorineural hearing loss. The goal of this research is to direct the differentiation of embryonic stem cells (SCs) into bipolar auditory neurons that can be used to replace degenerating neurons in the deafened mammalian cochlea. Successful replacement of auditory neurons is likely to result in improved clinical outcomes for cochlear implant recipients. We examined two post-natal auditory co-culture models with and without neurotrophic support, for their potential to direct the differentiation of mouse embryonic SCs into characteristic, bipolar, auditory neurons. The differentiation of SCs into neuron-like cells was facilitated by co-culture with auditory neurons or hair cell explants, isolated from post-natal day five rats. The most successful combination was the co-culture of hair cell explants with whole embryoid bodies, which resulted in significantly greater numbers of neurofilament-positive, neuron-like cells. While further characterization of these differentiated cells will be essential before transplantation studies commence, these data illustrate the effectiveness of post-natal hair cell explant co-culture, at providing valuable molecular cues for directed differentiation of SCs towards an auditory neuron lineage.     

Psychophysiology and psychoacoustics of music: Perception of complex sound in normal subjects and psychiatric patients

Perception of complex sound is a process carried out in everyday life situations and contributes in the way one perceives reality. Attempting to explain sound perception and how it affects human beings is complicated. Physics of simple sound can be described as a function of frequency, amplitude and phase. Psychology of sound, also termed psychoacoustics, has its own distinct elements of pitch, intensity and tibre. An interconnection exists between physics and psychology of hearing.Music being a complex sound contributes to communication and conveys information with semantic and emotional elements. These elements indicate the involvement of the central nervous system through processes of integration and interpretation together with peripheral auditory processing.Effects of sound and music in human psychology and physiology are complicated. Psychological influences of listening to different types of music are based on the different characteristics of basic musical sounds. Attempting to explain music perception can be simpler if music is broken down to its basic auditory signals. Perception of auditory signals is analyzed by the science of psychoacoustics. Differences in complex sound perception have been found between normal subjects and psychiatric patients and between different types of psychopathologies.     

Interaural Level Differences Do Not Suffice for Restoring Spatial Release from Masking in Simulated Cochlear Implant Listening

Spatial release from masking refers to a benefit for speech understanding. It occurs when a target talker and a masker talker are spatially separated. In those cases, speech intelligibility for target speech is typically higher than when both talkers are at the same location. In cochlear implant listeners, spatial release from masking is much reduced or absent compared with normal hearing listeners. Perhaps this reduced spatial release occurs because cochlear implant listeners cannot effectively attend to spatial cues. Three experiments examined factors that may interfere with deploying spatial attention to a target talker masked by another talker. To simulate cochlear implant listening, stimuli were vocoded with two unique features. First, we used 50-Hz low-pass filtered speech envelopes and noise carriers, strongly reducing the possibility of temporal pitch cues; second, co-modulation was imposed on target and masker utterances to enhance perceptual fusion between the two sources. Stimuli were presented over headphones. Experiments 1 and 2 presented high-fidelity spatial cues with unprocessed and vocoded speech. Experiment 3 maintained faithful long-term average interaural level differences but presented scrambled interaural time differences with vocoded speech. Results show a robust spatial release from masking in Experiments 1 and 2, and a greatly reduced spatial release in Experiment 3. Faithful long-term average interaural level differences were insufficient for producing spatial release from masking. This suggests that appropriate interaural time differences are necessary for restoring spatial release from masking, at least for a situation where there are few viable alternative segregation cues.     

Neurotrophin Gene Therapy for Sustained Neural Preservation after Deafness

The cochlear implant provides auditory cues to profoundly deaf patients by electrically stimulating the residual spiral ganglion neurons. These neurons, however, undergo progressive degeneration after hearing loss, marked initially by peripheral fibre retraction and ultimately culminating in cell death. This research aims to use gene therapy techniques to both hold and reverse this degeneration by providing a sustained and localised source of neurotrophins to the deafened cochlea. Adenoviral vectors containing green fluorescent protein, with or without neurotrophin-3 and brain derived neurotrophic factor, were injected into the lower basal turn of scala media of guinea pigs ototoxically deafened one week prior to intervention. This single injection resulted in localised and sustained gene expression, principally in the supporting cells within the organ of Corti. Guinea pigs treated with adenoviral neurotrophin-gene therapy had greater neuronal survival compared to contralateral non-treated cochleae when examined at 7 and 11 weeks post injection. Moreover; there was evidence of directed peripheral fibre regrowth towards cells expressing neurotrophin genes after both treatment periods. These data suggest that neurotrophin-gene therapy can provide sustained protection of spiral ganglion neurons and peripheral fibres after hearing loss.     

Auditory neuroplasticity,hearing loss and cochlear implants

Data from our laboratory show that the auditory brain is highly malleable by experience. We establish a base of knowledge that describes the normal structure and workings at the initial stages of the central auditory system. This research is expanded to include the associated pathology in the auditory brain stem created by hearing loss. Utilizing the congenitally deaf white cat, we demonstrate the way that cells, synapses, and circuits are pathologically affected by sound deprivation. We further show that the restoration of auditory nerve activity via electrical stimulation through cochlear implants serves to correct key features of brain pathology caused by hearing loss. The data suggest that rigorous training with cochlear implants and/or hearing aids offers the promise of heretofore unattained benefits.     

Precursors of Dancing and Singing to Music in Three- to Four-Months-Old Infants

Dancing and singing to music involve auditory-motor coordination and have been essential to our human culture since ancient times. Although scholars have been trying to understand the evolutionary and developmental origin of music, early human developmental manifestations of auditory-motor interactions in music have not been fully investigated. Here we report limb movements and vocalizations in three- to four-months-old infants while they listened to music and were in silence. In the group analysis, we found no significant increase in the amount of movement or in the relative power spectrum density around the musical tempo in the music condition compared to the silent condition. Intriguingly, however, there were two infants who demonstrated striking increases in the rhythmic movements via kicking or arm-waving around the musical tempo during listening to music. Monte-Carlo statistics with phase-randomized surrogate data revealed that the limb movements of these individuals were significantly synchronized to the musical beat. Moreover, we found a clear increase in the formant variability of vocalizations in the group during music perception. These results suggest that infants at this age are already primed with their bodies to interact with music via limb movements and vocalizations.