Dynamics of Vocalization-Induced Modulation of Auditory Cortical Activity at Mid-utterance

Background

Recent research has addressed the suppression of cortical sensory responses to altered auditory feedback that occurs at utterance onset regarding speech. However, there is reason to assume that the mechanisms underlying sensorimotor processing at mid-utterance are different than those involved in sensorimotor control at utterance onset. The present study attempted to examine the dynamics of event-related potentials (ERPs) to different acoustic versions of auditory feedback at mid-utterance.

Methodology/Principal findings

Subjects produced a vowel sound while hearing their pitch-shifted voice (100 cents), a sum of their vocalization and pure tones, or a sum of their vocalization and white noise at mid-utterance via headphones. Subjects also passively listened to playback of what they heard during active vocalization. Cortical ERPs were recorded in response to different acoustic versions of feedback changes during both active vocalization and passive listening. The results showed that, relative to passive listening, active vocalization yielded enhanced P2 responses to the 100 cents pitch shifts, whereas suppression effects of P2 responses were observed when voice auditory feedback was distorted by pure tones or white noise.

Conclusion/Significance

The present findings, for the first time, demonstrate a dynamic modulation of cortical activity as a function of the quality of acoustic feedback at mid-utterance, suggesting that auditory cortical responses can be enhanced or suppressed to distinguish self-produced speech from externally-produced sounds.     

Neural responses in human superior temporal cortex support coding of voice representations

The ability to recognize abstract features of voice during auditory perception is an intricate feat of human audition. For the listener, this occurs in near-automatic fashion to seamlessly extract complex cues from a highly variable auditory signal. Voice perception depends on specialized regions of auditory cortex, including superior temporal gyrus (STG) and superior temporal sulcus (STS). However, the nature of voice encoding at the cortical level remains poorly understood. We leverage intracerebral recordings across human auditory cortex during presentation of voice and nonvoice acoustic stimuli to examine voice encoding at the cortical level in 8 patient-participants undergoing epilepsy surgery evaluation. We show that voice selectivity increases along the auditory hierarchy from supratemporal plane (STP) to the STG and STS. Results show accurate decoding of vocalizations from human auditory cortical activity even in the complete absence of linguistic content. These findings show an early, less-selective temporal window of neural activity in the STG and STS followed by a sustained, strongly voice-selective window. Encoding models demonstrate divergence in the encoding of acoustic features along the auditory hierarchy, wherein STG/STS responses are best explained by voice category and acoustics, as opposed to acoustic features of voice stimuli alone. This is in contrast to neural activity recorded from STP, in which responses were accounted for by acoustic features. These findings support a model of voice perception that engages categorical encoding mechanisms within STG and STS to facilitate feature extraction.

Voice perception occurs via specialized networks in higher order auditory cortex, but how voice features are encoded remains a central unanswered question. Using human intracerebral recordings of auditory cortex, this study provides evidence for categorical encoding of voice.     

Neuronal mechanisms of voice control are affected by implicit expectancy of externally triggered perturbations in auditory feedback

Accurate vocal production relies on several factors including sensory feedback and the ability to predict future challenges to the control processes. Repetitive patterns of perturbations in sensory feedback by themselves elicit implicit expectations in the vocal control system regarding the timing, quality and direction of perturbations. In the present study, the predictability of voice pitch-shifted auditory feedback was experimentally manipulated. A block of trials where all pitch-shift stimuli were upward, and therefore predictable was contrasted against an unpredictable block of trials in which the stimulus direction was randomized between upward and downward pitch-shifts. It was found that predictable perturbations in voice auditory feedback led to a reduction in the proportion of compensatory vocal responses, which might be indicative of a reduction in vocal control. The predictable perturbations also led to a reduction in the magnitude of the N1 component of cortical Event Related Potentials (ERP) that was associated with the reflexive compensations to the perturbations. We hypothesize that formation of expectancy in our study is accompanied by involuntary allocation of attentional resources occurring as a result of habituation or learning, that in turn trigger limited and controlled exploration-related motor variability in the vocal control system.     

Human auditory cortical activation during self-vocalization

During speaking, auditory feedback is used to adjust vocalizations. The brain systems mediating this integrative ability have been investigated using a wide range of experimental strategies. In this report we examined how vocalization alters speech-sound processing within auditory cortex by directly recording evoked responses to vocalizations and playback stimuli using intracranial electrodes implanted in neurosurgery patients. Several new findings resulted from these high-resolution invasive recordings in human subjects. Suppressive effects of vocalization were found to occur only within circumscribed areas of auditory cortex. In addition, at a smaller number of sites, the opposite pattern was seen; cortical responses were enhanced during vocalization. This increase in activity was reflected in high gamma power changes, but was not evident in the averaged evoked potential waveforms. These new findings support forward models for vocal control in which efference copies of premotor cortex activity modulate sub-regions of auditory cortex.     

Vocal responses to perturbations in voice auditory feedback in individuals with Parkinson's disease

Background

One of the most common symptoms of speech deficits in individuals with Parkinson''s disease (PD) is significantly reduced vocal loudness and pitch range. The present study investigated whether abnormal vocalizations in individuals with PD are related to sensory processing of voice auditory feedback. Perturbations in loudness or pitch of voice auditory feedback are known to elicit short latency, compensatory responses in voice amplitude or fundamental frequency.

Methodology/Principal Findings

Twelve individuals with Parkinson''s disease and 13 age- and sex- matched healthy control subjects sustained a vowel sound (/α/) and received unexpected, brief (200 ms) perturbations in voice loudness (±3 or 6 dB) or pitch (±100 cents) auditory feedback. Results showed that, while all subjects produced compensatory responses in their voice amplitude or fundamental frequency, individuals with PD exhibited larger response magnitudes than the control subjects. Furthermore, for loudness-shifted feedback, upward stimuli resulted in shorter response latencies than downward stimuli in the control subjects but not in individuals with PD.

Conclusions/Significance

The larger response magnitudes in individuals with PD compared with the control subjects suggest that processing of voice auditory feedback is abnormal in PD. Although the precise mechanisms of the voice feedback processing are unknown, results of this study suggest that abnormal voice control in individuals with PD may be related to dysfunctional mechanisms of error detection or correction in sensory feedback processing.     

Auditory feedback control of vocal pitch during sustained vocalization: a cross-sectional study of adult aging

Background

Auditory feedback has been demonstrated to play an important role in the control of voice fundamental frequency (F₀), but the mechanisms underlying the processing of auditory feedback remain poorly understood. It has been well documented that young adults can use auditory feedback to stabilize their voice F₀ by making compensatory responses to perturbations they hear in their vocal pitch feedback. However, little is known about the effects of aging on the processing of audio-vocal feedback during vocalization.

Methodology/Principal Findings

In the present study, we recruited adults who were between 19 and 75 years of age and divided them into five age groups. Using a pitch-shift paradigm, the pitch of their vocal feedback was unexpectedly shifted ±50 or ±100 cents during sustained vocalization of the vowel sound/u/. Compensatory vocal F₀ response magnitudes and latencies to pitch feedback perturbations were examined. A significant effect of age was found such that response magnitudes increased with increasing age until maximal values were reached for adults 51–60 years of age and then decreased for adults 61–75 years of age. Adults 51–60 years of age were also more sensitive to the direction and magnitude of the pitch feedback perturbations compared to younger adults.

Conclusion

These findings demonstrate that the pitch-shift reflex systematically changes across the adult lifespan. Understanding aging-related changes to the role of auditory feedback is critically important for our theoretical understanding of speech production and the clinical applications of that knowledge.     

A cortical network processes auditory error signals during human speech production to maintain fluency

Hearing one’s own voice is critical for fluent speech production as it allows for the detection and correction of vocalization errors in real time. This behavior known as the auditory feedback control of speech is impaired in various neurological disorders ranging from stuttering to aphasia; however, the underlying neural mechanisms are still poorly understood. Computational models of speech motor control suggest that, during speech production, the brain uses an efference copy of the motor command to generate an internal estimate of the speech output. When actual feedback differs from this internal estimate, an error signal is generated to correct the internal estimate and update necessary motor commands to produce intended speech. We were able to localize the auditory error signal using electrocorticographic recordings from neurosurgical participants during a delayed auditory feedback (DAF) paradigm. In this task, participants hear their voice with a time delay as they produced words and sentences (similar to an echo on a conference call), which is well known to disrupt fluency by causing slow and stutter-like speech in humans. We observed a significant response enhancement in auditory cortex that scaled with the duration of feedback delay, indicating an auditory speech error signal. Immediately following auditory cortex, dorsal precentral gyrus (dPreCG), a region that has not been implicated in auditory feedback processing before, exhibited a markedly similar response enhancement, suggesting a tight coupling between the 2 regions. Critically, response enhancement in dPreCG occurred only during articulation of long utterances due to a continuous mismatch between produced speech and reafferent feedback. These results suggest that dPreCG plays an essential role in processing auditory error signals during speech production to maintain fluency.

Hearing one’s own voice is critical for fluent speech production, allowing detection and correction of vocalization errors in real-time. This study shows that the dorsal precentral gyrus is a critical component of a cortical network that monitors auditory feedback to produce fluent speech; this region is engaged specifically when speech production is effortful during articulation of long utterances.     

听觉皮层信号处理

听觉系统和视觉系统的不同之处在于：听觉系统在外周感受器和听皮层间具有更长的皮层下通路和更多的突触联系。该特殊结构反应了听觉系统从复杂听觉环境中提取与行为相关信号的机制与其他感觉系统不同。听皮层神经信号处理包括两种重要的转换机制,声音信号的非同构转换以及从声音感受到知觉层面的转换。听觉皮层神经编码机制同时也受到听觉反馈和语言或发声过程中发声信号的调控。听觉神经科学家和生物医学工程师所面临的挑战便是如何去理解大脑中这些转换的编码机制。我将会用我实验室最近的一些发现来阐述听觉信号是如何在原听皮层中进行处理的,并讨论其对于言语和音乐在大脑中的处理机制以及设计神经替代装置诸如电子耳蜗的意义。我们使用了结合神经电生理技术和量化工程学的方法来研究这些问题。     

LaDIVA: A neurocomputational model providing laryngeal motor control for speech acquisition and production

Many voice disorders are the result of intricate neural and/or biomechanical impairments that are poorly understood. The limited knowledge of their etiological and pathophysiological mechanisms hampers effective clinical management. Behavioral studies have been used concurrently with computational models to better understand typical and pathological laryngeal motor control. Thus far, however, a unified computational framework that quantitatively integrates physiologically relevant models of phonation with the neural control of speech has not been developed. Here, we introduce LaDIVA, a novel neurocomputational model with physiologically based laryngeal motor control. We combined the DIVA model (an established neural network model of speech motor control) with the extended body-cover model (a physics-based vocal fold model). The resulting integrated model, LaDIVA, was validated by comparing its model simulations with behavioral responses to perturbations of auditory vocal fundamental frequency (f_o) feedback in adults with typical speech. LaDIVA demonstrated capability to simulate different modes of laryngeal motor control, ranging from short-term (i.e., reflexive) and long-term (i.e., adaptive) auditory feedback paradigms, to generating prosodic contours in speech. Simulations showed that LaDIVA’s laryngeal motor control displays properties of motor equivalence, i.e., LaDIVA could robustly generate compensatory responses to reflexive vocal f_o perturbations with varying initial laryngeal muscle activation levels leading to the same output. The model can also generate prosodic contours for studying laryngeal motor control in running speech. LaDIVA can expand the understanding of the physiology of human phonation to enable, for the first time, the investigation of causal effects of neural motor control in the fine structure of the vocal signal.     

The processing of temporal pitch and melody information in auditory cortex

An fMRI experiment was performed to identify the main stages of melody processing in the auditory pathway. Spectrally matched sounds that produce no pitch, fixed pitch, or melody were all found to activate Heschl's gyrus (HG) and planum temporale (PT). Within this region, sounds with pitch produced more activation than those without pitch only in the lateral half of HG. When the pitch was varied to produce a melody, there was activation in regions beyond HG and PT, specifically in the superior temporal gyrus (STG) and planum polare (PP). The results support the view that there is hierarchy of pitch processing in which the center of activity moves anterolaterally away from primary auditory cortex as the processing of melodic sounds proceeds.     

Speech-evoked activity in primary auditory cortex: effects of voice onset time

Neural encoding of temporal speech features is a key component of acoustic and phonetic analyses. We examined the temporal encoding of the syllables /da/ and /ta/, which differ along the temporally based, phonetic parameter of voice onset time (VOT), in primary auditory cortex (A1) of awake monkeys using concurrent multilaminar recordings of auditory evoked potentials (AEP), the derived current source density, and multiunit activity. A general sequence of A1 activation consisting of a lamina-specific profile of parallel and sequential excitatory and inhibitory processes is described. VOT is encoded in the temporal response patterns of phase-locked activity to the periodic speech segments and by “on” responses to stimulus and voicing onset. A transformation occurs between responses in the thalamocortical (TC) fiber input and A1 cells. TC fibers are more likely to encode VOT with “on” responses to stimulus onset followed by phase-locked responses during the voiced segment, whereas A1 responses are more likely to exhibit transient responses both to stimulus and voicing onset. Relevance to subcortical speech processing, the human AEP and speech psychoacoustics are discussed. A mechanism for categorical differentiation of voiced and unvoiced consonants is proposed.     

Tuning to natural stimulus dynamics in primary auditory cortex

The amplitude and pitch fluctuations of natural soundscapes often exhibit "1/f spectra", which means that large, abrupt changes in pitch or loudness occur proportionally less frequently in nature than gentle, gradual fluctuations. Furthermore, human listeners reportedly prefer 1/f distributed random melodies to melodies with faster (1/f0) or slower (1/f2) dynamics. One might therefore suspect that neurons in the central auditory system may be tuned to 1/f dynamics, particularly given that recent reports provide evidence for tuning to 1/f dynamics in primary visual cortex. To test whether neurons in primary auditory cortex (A1) are tuned to 1/f dynamics, we recorded responses to random tone complexes in which the fundamental frequency and the envelope were determined by statistically independent "1/f(gamma) random walks," with gamma set to values between 0.5 and 4. Many A1 neurons showed clear evidence of tuning and responded with higher firing rates to stimuli with gamma between 1 and 1.5. Response patterns elicited by 1/f(gamma) stimuli were more reproducible for values of gamma close to 1. These findings indicate that auditory cortex is indeed tuned to the 1/f dynamics commonly found in the statistical distributions of natural soundscapes.     

An Adapting Auditory-motor Feedback Loop Can Contribute to Generating Vocal Repetition

Consecutive repetition of actions is common in behavioral sequences. Although integration of sensory feedback with internal motor programs is important for sequence generation, if and how feedback contributes to repetitive actions is poorly understood. Here we study how auditory feedback contributes to generating repetitive syllable sequences in songbirds. We propose that auditory signals provide positive feedback to ongoing motor commands, but this influence decays as feedback weakens from response adaptation during syllable repetitions. Computational models show that this mechanism explains repeat distributions observed in Bengalese finch song. We experimentally confirmed two predictions of this mechanism in Bengalese finches: removal of auditory feedback by deafening reduces syllable repetitions; and neural responses to auditory playback of repeated syllable sequences gradually adapt in sensory-motor nucleus HVC. Together, our results implicate a positive auditory-feedback loop with adaptation in generating repetitive vocalizations, and suggest sensory adaptation is important for feedback control of motor sequences.     

Human dorsal and ventral auditory streams subserve rehearsal-based and echoic processes during verbal working memory

To hear a sequence of words and repeat them requires sensory-motor processing and something more-temporary storage. We investigated neural mechanisms of verbal memory by using fMRI and a task designed to tease apart perceptually based ("echoic") memory from phonological-articulatory memory. Sets of two- or three-word pairs were presented bimodally, followed by a cue indicating from which modality (auditory or visual) items were to be retrieved and rehearsed over a delay. Although delay-period activation in the planum temporale (PT) was insensible to the source modality and showed sustained delay-period activity, the superior temporal gyrus (STG) activated more vigorously when the retrieved items had arrived to the auditory modality and showed transient delay-period activity. Functional connectivity analysis revealed two topographically distinct fronto-temporal circuits, with STG co-activating more strongly with ventrolateral prefrontal cortex and PT co-activating more strongly with dorsolateral prefrontal cortex. These argue for separate contributions of ventral and dorsal auditory streams in verbal working memory.     

A Lightweight,Headphones-based System for Manipulating Auditory Feedback in Songbirds

Experimental manipulations of sensory feedback during complex behavior have provided valuable insights into the computations underlying motor control and sensorimotor plasticity¹. Consistent sensory perturbations result in compensatory changes in motor output, reflecting changes in feedforward motor control that reduce the experienced feedback error. By quantifying how different sensory feedback errors affect human behavior, prior studies have explored how visual signals are used to recalibrate arm movements^2,3 and auditory feedback is used to modify speech production^4-7. The strength of this approach rests on the ability to mimic naturalistic errors in behavior, allowing the experimenter to observe how experienced errors in production are used to recalibrate motor output.Songbirds provide an excellent animal model for investigating the neural basis of sensorimotor control and plasticity^8,9. The songbird brain provides a well-defined circuit in which the areas necessary for song learning are spatially separated from those required for song production, and neural recording and lesion studies have made significant advances in understanding how different brain areas contribute to vocal behavior^9-12. However, the lack of a naturalistic error-correction paradigm - in which a known acoustic parameter is perturbed by the experimenter and then corrected by the songbird - has made it difficult to understand the computations underlying vocal learning or how different elements of the neural circuit contribute to the correction of vocal errors¹³.The technique described here gives the experimenter precise control over auditory feedback errors in singing birds, allowing the introduction of arbitrary sensory errors that can be used to drive vocal learning. Online sound-processing equipment is used to introduce a known perturbation to the acoustics of song, and a miniaturized headphones apparatus is used to replace a songbird''s natural auditory feedback with the perturbed signal in real time. We have used this paradigm to perturb the fundamental frequency (pitch) of auditory feedback in adult songbirds, providing the first demonstration that adult birds maintain vocal performance using error correction¹⁴. The present protocol can be used to implement a wide range of sensory feedback perturbations (including but not limited to pitch shifts) to investigate the computational and neurophysiological basis of vocal learning.     

The neuronal correlates of digits backward are revealed by voxel-based morphometry and resting-state functional connectivity analyses

Digits backward (DB) is a widely used neuropsychological measure that is believed to be a simple and effective index of the capacity of the verbal working memory. However, its neural correlates remain elusive. The aim of this study is to investigate the neural correlates of DB in 299 healthy young adults by combining voxel-based morphometry (VBM) and resting-state functional connectivity (rsFC) analyses. The VBM analysis showed positive correlations between the DB scores and the gray matter volumes in the right anterior superior temporal gyrus (STG), the right posterior STG, the left inferior frontal gyrus and the left Rolandic operculum, which are four critical areas in the auditory phonological loop of the verbal working memory. Voxel-based correlation analysis was then performed between the positive rsFCs of these four clusters and the DB scores. We found that the DB scores were positively correlated with the rsFCs within the salience network (SN), that is, between the right anterior STG, the dorsal anterior cingulate cortex and the right fronto-insular cortex. We also found that the DB scores were negatively correlated with the rsFC within an anti-correlation network of the SN, between the right posterior STG and the left posterior insula. Our findings suggest that DB performance is related to the structural and functional organizations of the brain areas that are involved in the auditory phonological loop and the SN.     

Intensity dependence of auditory evoked potentials in behaving cats

The intensity dependence of auditory evoked potentials (AEPs) recorded epidurally over the primary (AI) and secondary (AII) areas of the auditory cortex was studied in behaving cats during wakefulness, sleep and anesthesia. Four kHz tones of 50, 60, 70, and 80 dB SPL, presented in random order every 2 ± 0.2 s by a bone conductor, elicited clear changes of the AEP amplitudes with increasing stimulus intensity, but individual components displayed different responses curves. AEP components from the AI region showed saturation of their amplitude with stimulus intensity (P13, P34) or no amplitude increase (N19), while amplitude and intensity were linearly related in the AII area. The intensity dependence of the first positive component (P12/P13) was consistently stronger for the AEP recorded from the AI than from the AII area, while later components exhibited no difference between AI and AII. During slow wave sleep, the intensity dependence of this first positive component increased in the two areas, while that of later components decreased. Pentobarbital anesthesia abolished almost all later components and depressed the intensity dependence of the first positive component both in the AI and AII area. These results indicate that (1) clear intensity dependence of AEP exists in the cat auditory cortex and (2) this intensity dependence, especially that of the first positive AEP component, shares functional similarities to the human augmenting/reducing phenomenon in the auditory modality concerning regional differences and sleep-waking cycle.     

The role of auditory feedback in vocal learning and maintenance

Auditory experience is critical for the acquisition and maintenance of learned vocalizations in both humans and songbirds. Despite the central role of auditory feedback in vocal learning and maintenance, where and how auditory feedback affects neural circuits important to vocal control remain poorly understood. Recent studies of singing birds have uncovered neural mechanisms by which feedback perturbations affect vocal plasticity and also have identified feedback-sensitive neurons at or near sites of auditory and vocal motor interaction. Additionally, recent studies in marmosets have underscored that even in the absence of vocal learning, vocalization remains flexible in the face of changing acoustical environments, pointing to rapid interactions between auditory and vocal motor systems. Finally, recent studies show that a juvenile songbird's initial auditory experience of a song model has long-lasting effects on sensorimotor neurons important to vocalization, shedding light on how auditory memories and feedback interact to guide vocal learning.     

Vocal attractiveness and voluntarily pitch-shifted voices

Previous studies have found that using software to pitch shift people's voices can boost their perceived attractiveness to opposite-sex adults: men prefer women's voices when pitch-shifted up, and women prefer men's voices when pitch-shifted down. In this study, we sought to determine whether speakers could affect their perceived vocal attractiveness by voluntarily shifting their own voices to reach specific target pitches (+20 Hz or −20 Hz, a pitch increment that is based on prior research). Two sets of Chinese college students participated in the research: 115 who served as speakers whose voices were recorded, and 167 who served as raters who evaluated the speakers' voices. We found that when female speakers increased their pitch they were judged as more attractive to both opposite-sex and same-sex raters. An additional unexpected finding was that male speakers tended to rate other males who shifted their voice up in pitch as more attractive. These findings suggest that voluntary pitch shifts can affect attractiveness, but that they do not fully match the patterns observed when pitch shifting is done digitally.     

Weak Responses to Auditory Feedback Perturbation during Articulation in Persons Who Stutter: Evidence for Abnormal Auditory-Motor Transformation

Previous empirical observations have led researchers to propose that auditory feedback (the auditory perception of self-produced sounds when speaking) functions abnormally in the speech motor systems of persons who stutter (PWS). Researchers have theorized that an important neural basis of stuttering is the aberrant integration of auditory information into incipient speech motor commands. Because of the circumstantial support for these hypotheses and the differences and contradictions between them, there is a need for carefully designed experiments that directly examine auditory-motor integration during speech production in PWS. In the current study, we used real-time manipulation of auditory feedback to directly investigate whether the speech motor system of PWS utilizes auditory feedback abnormally during articulation and to characterize potential deficits of this auditory-motor integration. Twenty-one PWS and 18 fluent control participants were recruited. Using a short-latency formant-perturbation system, we examined participants' compensatory responses to unanticipated perturbation of auditory feedback of the first formant frequency during the production of the monophthong [ε]. The PWS showed compensatory responses that were qualitatively similar to the controls' and had close-to-normal latencies (～150 ms), but the magnitudes of their responses were substantially and significantly smaller than those of the control participants (by 47% on average, p<0.05). Measurements of auditory acuity indicate that the weaker-than-normal compensatory responses in PWS were not attributable to a deficit in low-level auditory processing. These findings are consistent with the hypothesis that stuttering is associated with functional defects in the inverse models responsible for the transformation from the domain of auditory targets and auditory error information into the domain of speech motor commands.