Directed attention eliminates 'change deafness' in complex auditory scenes

In natural environments that contain multiple sound sources, acoustic energy arising from the different sources sums to produce a single complex waveform at each of the listener's ears. The auditory system must segregate this waveform into distinct streams to permit identification of the objects from which the signals emanate [1]. Although the processes involved in stream segregation are now reasonably well understood [1, 2 and 3], little is known about the nature of our perception of complex auditory scenes. Here, we examined complex scene perception by having listeners detect a discrete change to an auditory scene comprising multiple concurrent naturalistic sounds. We found that listeners were remarkably poor at detecting the disappearance of an individual auditory object when listening to scenes containing more than four objects, but they performed near perfectly when their attention was directed to the identity of a potential change. In the absence of directed attention, this "change deafness" [4] was greater for objects arising from a common location in space than for objects separated in azimuth. Change deafness was also observed for changes in object location, suggesting that it may reflect a general effect of the dependence of human auditory perception on attention.     

Spike-timing-based computation in sound localization

Spike timing is precise in the auditory system and it has been argued that it conveys information about auditory stimuli, in particular about the location of a sound source. However, beyond simple time differences, the way in which neurons might extract this information is unclear and the potential computational advantages are unknown. The computational difficulty of this task for an animal is to locate the source of an unexpected sound from two monaural signals that are highly dependent on the unknown source signal. In neuron models consisting of spectro-temporal filtering and spiking nonlinearity, we found that the binaural structure induced by spatialized sounds is mapped to synchrony patterns that depend on source location rather than on source signal. Location-specific synchrony patterns would then result in the activation of location-specific assemblies of postsynaptic neurons. We designed a spiking neuron model which exploited this principle to locate a variety of sound sources in a virtual acoustic environment using measured human head-related transfer functions. The model was able to accurately estimate the location of previously unknown sounds in both azimuth and elevation (including front/back discrimination) in a known acoustic environment. We found that multiple representations of different acoustic environments could coexist as sets of overlapping neural assemblies which could be associated with spatial locations by Hebbian learning. The model demonstrates the computational relevance of relative spike timing to extract spatial information about sources independently of the source signal.     

Object localization in cluttered acoustical environments

In nature, sounds from objects of interest arrive at the ears accompanied by sound waves from other actively emitting objects and by reflections off of nearby surfaces. Despite the fact that all of these waveforms sum at the eardrums, humans with normal hearing effortlessly segregate one sound source from another. Our laboratory is investigating the neural basis of this perceptual feat, often called the "cocktail party effect", using the barn owl as an animal model. The barn owl, renowned for its ability to localize sounds and its spatiotopic representation of auditory space, is an established model for spatial hearing. Here, we briefly review the neural basis of sound-localization of a single sound source in an anechoic environment and then generalize the ideas developed therein to cases in which there are multiple, concomitant sound sources and acoustical reflection.     

Constructing Noise-Invariant Representations of Sound in the Auditory Pathway

Identifying behaviorally relevant sounds in the presence of background noise is one of the most important and poorly understood challenges faced by the auditory system. An elegant solution to this problem would be for the auditory system to represent sounds in a noise-invariant fashion. Since a major effect of background noise is to alter the statistics of the sounds reaching the ear, noise-invariant representations could be promoted by neurons adapting to stimulus statistics. Here we investigated the extent of neuronal adaptation to the mean and contrast of auditory stimulation as one ascends the auditory pathway. We measured these forms of adaptation by presenting complex synthetic and natural sounds, recording neuronal responses in the inferior colliculus and primary fields of the auditory cortex of anaesthetized ferrets, and comparing these responses with a sophisticated model of the auditory nerve. We find that the strength of both forms of adaptation increases as one ascends the auditory pathway. To investigate whether this adaptation to stimulus statistics contributes to the construction of noise-invariant sound representations, we also presented complex, natural sounds embedded in stationary noise, and used a decoding approach to assess the noise tolerance of the neuronal population code. We find that the code for complex sounds in the periphery is affected more by the addition of noise than the cortical code. We also find that noise tolerance is correlated with adaptation to stimulus statistics, so that populations that show the strongest adaptation to stimulus statistics are also the most noise-tolerant. This suggests that the increase in adaptation to sound statistics from auditory nerve to midbrain to cortex is an important stage in the construction of noise-invariant sound representations in the higher auditory brain.     

The amplitude sensitivity of mouse inferior collicular neurons in the presence of weak noise

Natural auditory environment consists of multiple sound sources that are embedded in ambient strong and weak noise. For effective sound communication and signal analysis, animals must somehow extract biologically relevant signals from the inevitable interference of ambient noise. The present study examined how a weak noise may affect the amplitude sensitivity of neurons in the mouse central nucleus of the inferior colliculus (IC) which receives convergent excitatory and inhibitory inputs from both lower and higher auditory centers. Specifically, we studied the amplitude sensitivity of IC neurons using a probe (best frequency pulse) and a masker (weak noise) under simultaneous masking paradigm. For most IC neurons, weak noise masking increases the minimum threshold and decreases the number of impulses. Noise masking also increased the slope and decreased the dynamic range of the rate amplitude function of these IC neurons. The strength of this noise masking was greater at low than at high sound amplitudes. This variation in the amplitude sensitivity of IC neurons in the presence of the weak noise was mostly mediated through GABAergic inhibition. These data indicate that in the real world the ambient weak noise improves amplitude sensitivity of IC neurons through GABAergic inhibition while inevitably decreases the range of overall auditory sensitivity of IC neurons.     

Effects of noise bandwidth and amplitude modulation on masking in frog auditory midbrain neurons

Natural auditory scenes such as frog choruses consist of multiple sound sources (i.e., individual vocalizing males) producing sounds that overlap extensively in time and spectrum, often in the presence of other biotic and abiotic background noise. Detection of a signal in such environments is challenging, but it is facilitated when the noise shares common amplitude modulations across a wide frequency range, due to a phenomenon called comodulation masking release (CMR). Here, we examined how properties of the background noise, such as its bandwidth and amplitude modulation, influence the detection threshold of a target sound (pulsed amplitude modulated tones) by single neurons in the frog auditory midbrain. We found that for both modulated and unmodulated masking noise, masking was generally stronger with increasing bandwidth, but it was weakened for the widest bandwidths. Masking was less for modulated noise than for unmodulated noise for all bandwidths. However, responses were heterogeneous, and only for a subpopulation of neurons the detection of the probe was facilitated when the bandwidth of the modulated masker was increased beyond a certain bandwidth - such neurons might contribute to CMR. We observed evidence that suggests that the dips in the noise amplitude are exploited by TS neurons, and observed strong responses to target signals occurring during such dips. However, the interactions between the probe and masker responses were nonlinear, and other mechanisms, e.g., selective suppression of the response to the noise, may also be involved in the masking release.     

听觉中枢神经元对声信号的识别和处理

人类听觉的基本特性和机制与其他哺乳动物相似,因此,利用动物所作的听觉研究和获得的结果,有助于认识人类自身的听觉.围绕听觉中枢神经元对不同模式的声信号的识别和处理,简要综述了这方面的研究.声信号和声模式识别在听觉中枢对声信号的感受和加工中具有重要意义.听神经元作为声模式识别的结构和功能基础,对不同的声刺激模式产生不同反应,甚至是在同一声刺激模式下,改变其中的某个声参数,神经元的反应也会发生相应改变,而其反应的特性和机制均需要更多研究来解答.另外,声信号作为声信息的载体,不同的声信息寓于不同的声参数和声特征之中,研究发现,听觉中枢神经元存在相应的声信息甄别和选择的神经基础,能对动态变化的声频率、幅度和时程等进行反应和编码,并且,在不同种类动物上获得的研究结果极为相似,表明听觉中枢对不同声信号和声刺激模式的识别、分析和加工,具有共同性和普遍性.     

Directional hearing in a silicon cricket

Phonotaxis is the ability to orient towards or away from sound sources. Crickets can locate conspecifics by phonotaxis to the calling (mating) song they produce, and can evade bats by negative phonotaxis from echolocation calls. The behaviour and underlying physiology have been studied in some depth, and the auditory system solves this complex problem in a unique manner. Experiments conducted on a simulation model of the system indicated that the mechanism output a directional signal to sounds ahead at calling song frequency and to sounds behind at echolocation frequencies. We suggest that this combination of responses helps simplify later processing in the cricket. To further explore this result, an analogue, very large scale integrated (aVLSI) circuit model of the mechanism was designed and built; results from testing this agreed with the simulation. The aVLSI circuit was used to test a further hypothesis about the potential advantages of the positioning of the acoustic inputs for sound localisation during walking. There was no clear advantage to the directionality of the system in their location. The aVLSI circuitry is now being extended to use on a robot along with previously modelled neural circuitry to better understand the complete sensorimotor pathway.     

Philosophy and stimulus design for neuroethology of complex-sound processing.

In research on the neural mechanisms for the processing of biologically important sounds such as species-specific sounds and sounds produced by prey and predators, it is necessary to study responses of central auditory neurons to biologically important sounds, information-bearing elements (IBEs) in them, and tone bursts. The tone bursts or constant-frequency (CF) components can be an IBE in many species of animals. Information-bearing parameters characterizing these sounds must be systematically varied, and tuning of neurons to individual parameters must be studied. The measurement of a tuning curve must be performed not only for excitatory responses, but also for inhibitory and facilitative responses, if any. The selectivity of a neuron to a particular type of sound must be tested for whether it is level-tolerant. Responses to complex sounds can probably be explained on the basis of those to IBEs and tone bursts, so that the use of the tone bursts, even though they are not IBEs, is as essential as that of the biologically important sounds.     

The Opponent Channel Population Code of Sound Location Is an Efficient Representation of Natural Binaural Sounds

In mammalian auditory cortex, sound source position is represented by a population of broadly tuned neurons whose firing is modulated by sounds located at all positions surrounding the animal. Peaks of their tuning curves are concentrated at lateral position, while their slopes are steepest at the interaural midline, allowing for the maximum localization accuracy in that area. These experimental observations contradict initial assumptions that the auditory space is represented as a topographic cortical map. It has been suggested that a “panoramic” code has evolved to match specific demands of the sound localization task. This work provides evidence suggesting that properties of spatial auditory neurons identified experimentally follow from a general design principle- learning a sparse, efficient representation of natural stimuli. Natural binaural sounds were recorded and served as input to a hierarchical sparse-coding model. In the first layer, left and right ear sounds were separately encoded by a population of complex-valued basis functions which separated phase and amplitude. Both parameters are known to carry information relevant for spatial hearing. Monaural input converged in the second layer, which learned a joint representation of amplitude and interaural phase difference. Spatial selectivity of each second-layer unit was measured by exposing the model to natural sound sources recorded at different positions. Obtained tuning curves match well tuning characteristics of neurons in the mammalian auditory cortex. This study connects neuronal coding of the auditory space with natural stimulus statistics and generates new experimental predictions. Moreover, results presented here suggest that cortical regions with seemingly different functions may implement the same computational strategy-efficient coding.     

Eye position influences auditory responses in primate inferior colliculus

We examined the frame of reference of auditory responses in the inferior colliculus in monkeys fixating visual stimuli at different locations. Eye position modulated the level of auditory responses in 33% of the neurons we encountered, but it did not appear to shift their spatial tuning. The effect of eye position on auditory responses was substantial-comparable in magnitude to that of sound location. The eye position signal appeared to interact with the auditory responses in at least a partly multiplicative fashion. We conclude that the representation of sound location in primate IC is distributed and that the frame of reference is intermediate between head- and eye-centered coordinates. The information contained in these neurons appears to be sufficient for later neural stages to calculate the positions of sounds with respect to the eyes.     

A Neural Model of Auditory Space Compatible with Human Perception under Simulated Echoic Conditions

In a typical auditory scene, sounds from different sources and reflective surfaces summate in the ears, causing spatial cues to fluctuate. Prevailing hypotheses of how spatial locations may be encoded and represented across auditory neurons generally disregard these fluctuations and must therefore invoke additional mechanisms for detecting and representing them. Here, we consider a different hypothesis in which spatial perception corresponds to an intermediate or sub-maximal firing probability across spatially selective neurons within each hemisphere. The precedence or Haas effect presents an ideal opportunity for examining this hypothesis, since the temporal superposition of an acoustical reflection with sounds arriving directly from a source can cause otherwise stable cues to fluctuate. Our findings suggest that subjects’ experiences may simply reflect the spatial cues that momentarily arise under various acoustical conditions and how these cues are represented. We further suggest that auditory objects may acquire “edges” under conditions when interaural time differences are broadly distributed.     

A Population Rate Code of Auditory Space in the Human Cortex

Background

Previous work on the human auditory cortex has revealed areas specialized in spatial processing but how the neurons in these areas represent the location of a sound source remains unknown.

Methodology/Principal Findings

Here, we performed a magnetoencephalography (MEG) experiment with the aim of revealing the neural code of auditory space implemented by the human cortex. In a stimulus-specific adaptation paradigm, realistic spatial sound stimuli were presented in pairs of adaptor and probe locations. We found that the attenuation of the N1m response depended strongly on the spatial arrangement of the two sound sources. These location-specific effects showed that sounds originating from locations within the same hemifield activated the same neuronal population regardless of the spatial separation between the sound sources. In contrast, sounds originating from opposite hemifields activated separate groups of neurons.

Conclusions/Significance

These results are highly consistent with a rate code of spatial location formed by two opponent populations, one tuned to locations in the left and the other to those in the right. This indicates that the neuronal code of sound source location implemented by the human auditory cortex is similar to that previously found in other primates.     

Neural representation of spectral and temporal information in speech

Speech is the most interesting and one of the most complex sounds dealt with by the auditory system. The neural representation of speech needs to capture those features of the signal on which the brain depends in language communication. Here we describe the representation of speech in the auditory nerve and in a few sites in the central nervous system from the perspective of the neural coding of important aspects of the signal. The representation is tonotopic, meaning that the speech signal is decomposed by frequency and different frequency components are represented in different populations of neurons. Essential to the representation are the properties of frequency tuning and nonlinear suppression. Tuning creates the decomposition of the signal by frequency, and nonlinear suppression is essential for maintaining the representation across sound levels. The representation changes in central auditory neurons by becoming more robust against changes in stimulus intensity and more transient. However, it is probable that the form of the representation at the auditory cortex is fundamentally different from that at lower levels, in that stimulus features other than the distribution of energy across frequency are analysed.     

Representation of Sound Objects within Early-Stage Auditory Areas: A Repetition Effect Study Using 7T fMRI

Environmental sounds are highly complex stimuli whose recognition depends on the interaction of top-down and bottom-up processes in the brain. Their semantic representations were shown to yield repetition suppression effects, i. e. a decrease in activity during exposure to a sound that is perceived as belonging to the same source as a preceding sound. Making use of the high spatial resolution of 7T fMRI we have investigated the representations of sound objects within early-stage auditory areas on the supratemporal plane. The primary auditory cortex was identified by means of tonotopic mapping and the non-primary areas by comparison with previous histological studies. Repeated presentations of different exemplars of the same sound source, as compared to the presentation of different sound sources, yielded significant repetition suppression effects within a subset of early-stage areas. This effect was found within the right hemisphere in primary areas A1 and R as well as two non-primary areas on the antero-medial part of the planum temporale, and within the left hemisphere in A1 and a non-primary area on the medial part of Heschl’s gyrus. Thus, several, but not all early-stage auditory areas encode the meaning of environmental sounds.     

Modeling complex tone perception: grouping harmonics with combination-sensitive neurons

 Perception of complex communication sounds is a major function of the auditory system. To create a coherent percept of these sounds the auditory system may instantaneously group or bind multiple harmonics within complex sounds. This perception strategy simplifies further processing of complex sounds and facilitates their meaningful integration with other sensory inputs. Based on experimental data and a realistic model, we propose that associative learning of combinations of harmonic frequencies and nonlinear facilitation of responses to those combinations, also referred to as “combination-sensitivity,” are important for spectral grouping. For our model, we simulated combination sensitivity using Hebbian and associative types of synaptic plasticity in auditory neurons. We also provided a parallel tonotopic input that converges and diverges within the network. Neurons in higher-order layers of the network exhibited an emergent property of multifrequency tuning that is consistent with experimental findings. Furthermore, this network had the capacity to “recognize” the pitch or fundamental frequency of a harmonic tone complex even when the fundamental frequency itself was missing. Received: 6 October 2001 / Accepted in revised form: 21 January 2002     

Sparse codes for speech predict spectrotemporal receptive fields in the inferior colliculus

We have developed a sparse mathematical representation of speech that minimizes the number of active model neurons needed to represent typical speech sounds. The model learns several well-known acoustic features of speech such as harmonic stacks, formants, onsets and terminations, but we also find more exotic structures in the spectrogram representation of sound such as localized checkerboard patterns and frequency-modulated excitatory subregions flanked by suppressive sidebands. Moreover, several of these novel features resemble neuronal receptive fields reported in the Inferior Colliculus (IC), as well as auditory thalamus and cortex, and our model neurons exhibit the same tradeoff in spectrotemporal resolution as has been observed in IC. To our knowledge, this is the first demonstration that receptive fields of neurons in the ascending mammalian auditory pathway beyond the auditory nerve can be predicted based on coding principles and the statistical properties of recorded sounds.     

Reverberation challenges the temporal representation of the pitch of complex sounds

Accurate neural coding of the pitch of complex sounds is an essential part of auditory scene analysis; differences in pitch help segregate concurrent sounds, while similarities in pitch can help group sounds from a common source. In quiet, nonreverberant backgrounds, pitch can be derived from timing information in broadband high-frequency auditory channels and/or from frequency and timing information carried in narrowband low-frequency auditory channels. Recording from single neurons in the cochlear nucleus of anesthetized guinea pigs, we show that the neural representation of pitch based on timing information is severely degraded in the presence of reverberation. This degradation increases with both increasing reverberation strength and channel bandwidth. In a parallel human psychophysical pitch-discrimination task, reverberation impaired the ability to distinguish a high-pass harmonic sound from noise. Together, these findings explain the origin of perceptual difficulties experienced by both normal-hearing and hearing-impaired listeners in reverberant spaces.     

Auditory Sketches: Very Sparse Representations of Sounds Are Still Recognizable

Sounds in our environment like voices, animal calls or musical instruments are easily recognized by human listeners. Understanding the key features underlying this robust sound recognition is an important question in auditory science. Here, we studied the recognition by human listeners of new classes of sounds: acoustic and auditory sketches, sounds that are severely impoverished but still recognizable. Starting from a time-frequency representation, a sketch is obtained by keeping only sparse elements of the original signal, here, by means of a simple peak-picking algorithm. Two time-frequency representations were compared: a biologically grounded one, the auditory spectrogram, which simulates peripheral auditory filtering, and a simple acoustic spectrogram, based on a Fourier transform. Three degrees of sparsity were also investigated. Listeners were asked to recognize the category to which a sketch sound belongs: singing voices, bird calls, musical instruments, and vehicle engine noises. Results showed that, with the exception of voice sounds, very sparse representations of sounds (10 features, or energy peaks, per second) could be recognized above chance. No clear differences could be observed between the acoustic and the auditory sketches. For the voice sounds, however, a completely different pattern of results emerged, with at-chance or even below-chance recognition performances, suggesting that the important features of the voice, whatever they are, were removed by the sketch process. Overall, these perceptual results were well correlated with a model of auditory distances, based on spectro-temporal excitation patterns (STEPs). This study confirms the potential of these new classes of sounds, acoustic and auditory sketches, to study sound recognition.     

Location coding by opponent neural populations in the auditory cortex

Although the auditory cortex plays a necessary role in sound localization, physiological investigations in the cortex reveal inhomogeneous sampling of auditory space that is difficult to reconcile with localization behavior under the assumption of local spatial coding. Most neurons respond maximally to sounds located far to the left or right side, with few neurons tuned to the frontal midline. Paradoxically, psychophysical studies show optimal spatial acuity across the frontal midline. In this paper, we revisit the problem of inhomogeneous spatial sampling in three fields of cat auditory cortex. In each field, we confirm that neural responses tend to be greatest for lateral positions, but show the greatest modulation for near-midline source locations. Moreover, identification of source locations based on cortical responses shows sharp discrimination of left from right but relatively inaccurate discrimination of locations within each half of space. Motivated by these findings, we explore an opponent-process theory in which sound-source locations are represented by differences in the activity of two broadly tuned channels formed by contra- and ipsilaterally preferring neurons. Finally, we demonstrate a simple model, based on spike-count differences across cortical populations, that provides bias-free, level-invariant localization—and thus also a solution to the “binding problem” of associating spatial information with other nonspatial attributes of sounds.