Acoustic-Emergent Phonology in the Amplitude Envelope of Child-Directed Speech

When acquiring language, young children may use acoustic spectro-temporal patterns in speech to derive phonological units in spoken language (e.g., prosodic stress patterns, syllables, phonemes). Children appear to learn acoustic-phonological mappings rapidly, without direct instruction, yet the underlying developmental mechanisms remain unclear. Across different languages, a relationship between amplitude envelope sensitivity and phonological development has been found, suggesting that children may make use of amplitude modulation (AM) patterns within the envelope to develop a phonological system. Here we present the Spectral Amplitude Modulation Phase Hierarchy (S-AMPH) model, a set of algorithms for deriving the dominant AM patterns in child-directed speech (CDS). Using Principal Components Analysis, we show that rhythmic CDS contains an AM hierarchy comprising 3 core modulation timescales. These timescales correspond to key phonological units: prosodic stress (Stress AM, ~2 Hz), syllables (Syllable AM, ~5 Hz) and onset-rime units (Phoneme AM, ~20 Hz). We argue that these AM patterns could in principle be used by naïve listeners to compute acoustic-phonological mappings without lexical knowledge. We then demonstrate that the modulation statistics within this AM hierarchy indeed parse the speech signal into a primitive hierarchically-organised phonological system comprising stress feet (proto-words), syllables and onset-rime units. We apply the S-AMPH model to two other CDS corpora, one spontaneous and one deliberately-timed. The model accurately identified 72–82% (freely-read CDS) and 90–98% (rhythmically-regular CDS) stress patterns, syllables and onset-rime units. This in-principle demonstration that primitive phonology can be extracted from speech AMs is termed Acoustic-Emergent Phonology (AEP) theory. AEP theory provides a set of methods for examining how early phonological development is shaped by the temporal modulation structure of speech across languages. The S-AMPH model reveals a crucial developmental role for stress feet (AMs ~2 Hz). Stress feet underpin different linguistic rhythm typologies, and speech rhythm underpins language acquisition by infants in all languages.     

EEG Source Reconstruction Reveals Frontal-Parietal Dynamics of Spatial Conflict Processing

Cognitive control requires the suppression of distracting information in order to focus on task-relevant information. We applied EEG source reconstruction via time-frequency linear constrained minimum variance beamforming to help elucidate the neural mechanisms involved in spatial conflict processing. Human subjects performed a Simon task, in which conflict was induced by incongruence between spatial location and response hand. We found an early (∼200 ms post-stimulus) conflict modulation in stimulus-contralateral parietal gamma (30–50 Hz), followed by a later alpha-band (8–12 Hz) conflict modulation, suggesting an early detection of spatial conflict and inhibition of spatial location processing. Inter-regional connectivity analyses assessed via cross-frequency coupling of theta (4–8 Hz), alpha, and gamma power revealed conflict-induced shifts in cortical network interactions: Congruent trials (relative to incongruent trials) had stronger coupling between frontal theta and stimulus-contrahemifield parietal alpha/gamma power, whereas incongruent trials had increased theta coupling between medial frontal and lateral frontal regions. These findings shed new light into the large-scale network dynamics of spatial conflict processing, and how those networks are shaped by oscillatory interactions.     

Dynamic large-scale connectivity of intrinsic cortical oscillations supports adaptive listening in challenging conditions

In multi-talker situations, individuals adapt behaviorally to this listening challenge mostly with ease, but how do brain neural networks shape this adaptation? We here establish a long-sought link between large-scale neural communications in electrophysiology and behavioral success in the control of attention in difficult listening situations. In an age-varying sample of N = 154 individuals, we find that connectivity between intrinsic neural oscillations extracted from source-reconstructed electroencephalography is regulated according to the listener’s goal during a challenging dual-talker task. These dynamics occur as spatially organized modulations in power-envelope correlations of alpha and low-beta neural oscillations during approximately 2-s intervals most critical for listening behavior relative to resting-state baseline. First, left frontoparietal low-beta connectivity (16 to 24 Hz) increased during anticipation and processing of a spatial-attention cue before speech presentation. Second, posterior alpha connectivity (7 to 11 Hz) decreased during comprehension of competing speech, particularly around target-word presentation. Connectivity dynamics of these networks were predictive of individual differences in the speed and accuracy of target-word identification, respectively, but proved unconfounded by changes in neural oscillatory activity strength. Successful adaptation to a listening challenge thus latches onto two distinct yet complementary neural systems: a beta-tuned frontoparietal network enabling the flexible adaptation to attentive listening state and an alpha-tuned posterior network supporting attention to speech.

This study investigates how intrinsic neural oscillations, acting in concert, tune into attentive listening. Using electroencephalography signals collected from people in a dual-talker listening task, the authors find that network connectivity of frontoparietal beta and posterior alpha oscillations is regulated according to the listener’s goal.     

High Gamma Oscillations in Medial Temporal Lobe during Overt Production of Speech and Gestures

The study of the production of co-speech gestures (CSGs), i.e., meaningful hand movements that often accompany speech during everyday discourse, provides an important opportunity to investigate the integration of language, action, and memory because of the semantic overlap between gesture movements and speech content. Behavioral studies of CSGs and speech suggest that they have a common base in memory and predict that overt production of both speech and CSGs would be preceded by neural activity related to memory processes. However, to date the neural correlates and timing of CSG production are still largely unknown. In the current study, we addressed these questions with magnetoencephalography and a semantic association paradigm in which participants overtly produced speech or gesture responses that were either meaningfully related to a stimulus or not. Using spectral and beamforming analyses to investigate the neural activity preceding the responses, we found a desynchronization in the beta band (15–25 Hz), which originated 900 ms prior to the onset of speech and was localized to motor and somatosensory regions in the cortex and cerebellum, as well as right inferior frontal gyrus. Beta desynchronization is often seen as an indicator of motor processing and thus reflects motor activity related to the hand movements that gestures add to speech. Furthermore, our results show oscillations in the high gamma band (50–90 Hz), which originated 400 ms prior to speech onset and were localized to the left medial temporal lobe. High gamma oscillations have previously been found to be involved in memory processes and we thus interpret them to be related to contextual association of semantic information in memory. The results of our study show that high gamma oscillations in medial temporal cortex play an important role in the binding of information in human memory during speech and CSG production.     

Effects of speech clarity on recognition memory for spoken sentences

Extensive research shows that inter-talker variability (i.e., changing the talker) affects recognition memory for speech signals. However, relatively little is known about the consequences of intra-talker variability (i.e. changes in speaking style within a talker) on the encoding of speech signals in memory. It is well established that speakers can modulate the characteristics of their own speech and produce a listener-oriented, intelligibility-enhancing speaking style in response to communication demands (e.g., when speaking to listeners with hearing impairment or non-native speakers of the language). Here we conducted two experiments to examine the role of speaking style variation in spoken language processing. First, we examined the extent to which clear speech provided benefits in challenging listening environments (i.e. speech-in-noise). Second, we compared recognition memory for sentences produced in conversational and clear speaking styles. In both experiments, semantically normal and anomalous sentences were included to investigate the role of higher-level linguistic information in the processing of speaking style variability. The results show that acoustic-phonetic modifications implemented in listener-oriented speech lead to improved speech recognition in challenging listening conditions and, crucially, to a substantial enhancement in recognition memory for sentences.     

From Mind to Mouth: Event Related Potentials of Sentence Production in Classic Galactosemia

Patients with classic galactosemia, an inborn error of metabolism, have speech and language production impairments. Past research primarily focused on speech (motor) problems, but these cannot solely explain the language impairments. Which specific deficits contribute to the impairments in language production is not yet known. Deficits in semantic and syntactic planning are plausible and require further investigation. In the present study, we examined syntactic encoding while patients and matched controls overtly described scenes of moving objects using either separate words (minimal syntactic planning) or sentences (sentence-level syntactic planning). The design of the paradigm also allowed tapping into local noun phrase- and more global sentence-level syntactic planning. Simultaneously, we recorded event-related potentials (ERPs). The patients needed more time to prepare and finish the utterances and made more errors. The patient ERPs had a very similar morphology to that of healthy controls, indicating overall comparable neural processing. Most importantly, the ERPs diverged from those of controls in several functionally informative time windows, ranging from very early (90–150 ms post scene onset) to relatively late (1820–2020 ms post scene onset). These time windows can be associated with different linguistic encoding stages. The ERP results form the first neuroscientific evidence for language production impairments in patients with galactosemia in lexical and syntactic planning stages, i.e., prior to the linguistic output phase. These findings hence shed new light on the language impairments in this disease.     

Role of joint language control during cross-language communication: evidence from cross-frequency coupling

How do bilingual interlocutors inhibit interference from the non-target language to achieve brain-to-brain information exchange in a task to simulate a bilingual speaker–listener interaction. In the current study, two electroencephalogram devices were employed to record pairs of participants’ performances in a joint language switching task. Twenty-eight (14 pairs) unbalanced Chinese–English bilinguals (L1 Chinese) were instructed to name pictures in the appropriate language according to the cue. The phase-amplitude coupling analysis was employed to reveal the large-scale brain network responsible for joint language control between interlocutors. We found that (1) speakers and listeners coordinately suppressed cross-language interference through cross-frequency coupling, as shown in the increased delta/theta phase-amplitude and delta/alpha phase-amplitude coupling when switching to L2 than switching to L1; (2) speakers and listeners were both able to simultaneously inhibit cross-person item-level interference which was demonstrated by stronger cross-frequency coupling in the cross person condition compared to the within person condition. These results indicate that current bilingual models (e.g., the inhibitory control model) should incorporate mechanisms that address inhibiting interference sourced in both language and person (i.e., cross-language and cross-person item-level interference) synchronously through joint language control in dynamic cross-language communication.     

A Corpus Investigation of Syntactic Embedding in Pirah?

The Pirahã language has been at the center of recent debates in linguistics, in large part because it is claimed not to exhibit recursion, a purported universal of human language. Here, we present an analysis of a novel corpus of natural Pirahã speech that was originally collected by Dan Everett and Steve Sheldon. We make the corpus freely available for further research. In the corpus, Pirahã sentences have been shallowly parsed and given morpheme-aligned English translations. We use the corpus to investigate the formal complexity of Pirahã syntax by searching for evidence of syntactic embedding. In particular, we search for sentences which could be analyzed as containing center-embedding, sentential complements, adverbials, complementizers, embedded possessors, conjunction or disjunction. We do not find unambiguous evidence for recursive embedding of sentences or noun phrases in the corpus. We find that the corpus is plausibly consistent with an analysis of Pirahã as a regular language, although this is not the only plausible analysis.     

Tuning of Human Modulation Filters Is Carrier-Frequency Dependent

Recent studies employing speech stimuli to investigate ‘cocktail-party’ listening have focused on entrainment of cortical activity to modulations at syllabic (5 Hz) and phonemic (20 Hz) rates. The data suggest that cortical modulation filters (CMFs) are dependent on the sound-frequency channel in which modulations are conveyed, potentially underpinning a strategy for separating speech from background noise. Here, we characterize modulation filters in human listeners using a novel behavioral method. Within an ‘inverted’ adaptive forced-choice increment detection task, listening level was varied whilst contrast was held constant for ramped increments with effective modulation rates between 0.5 and 33 Hz. Our data suggest that modulation filters are tonotopically organized (i.e., vary along the primary, frequency-organized, dimension). This suggests that the human auditory system is optimized to track rapid (phonemic) modulations at high sound-frequencies and slow (prosodic/syllabic) modulations at low frequencies.     

Motor Network Plasticity and Low-Frequency Oscillations Abnormalities in Patients with Brain Gliomas: A Functional MRI Study

Brain plasticity is often associated with the process of slow-growing tumor formation, which remodels neural organization and optimizes brain network function. In this study, we aimed to investigate whether motor function plasticity would display deficits in patients with slow-growing brain tumors located in or near motor areas, but who were without motor neurological deficits. We used resting-state functional magnetic resonance imaging to probe motor networks in 15 patients with histopathologically confirmed brain gliomas and 15 age-matched healthy controls. All subjects performed a motor task to help identify individual motor activity in the bilateral primary motor cortex (PMC) and supplementary motor area (SMA). Frequency-based analysis at three different frequencies was then used to investigate possible alterations in the power spectral density (PSD) of low-frequency oscillations. For each group, the average PSD was determined for each brain region and a nonparametric test was performed to determine the difference in power between the two groups. Significantly reduced inter-hemispheric functional connectivity between the left and right PMC was observed in patients compared with controls (P<0.05). We also found significantly decreased PSD in patients compared to that in controls, in all three frequency bands (low: 0.01–0.02 Hz; middle: 0.02–0.06 Hz; and high: 0.06–0.1 Hz), at three key motor regions. These findings suggest that in asymptomatic patients with brain tumors located in eloquent regions, inter-hemispheric connection may be more vulnerable. A comparison of the two approaches indicated that power spectral analysis is more sensitive than functional connectivity analysis for identifying the neurological abnormalities underlying motor function plasticity induced by slow-growing tumors.     

When Correction Turns Positive: Processing Corrective Prosody in Dutch

Current research on spoken language does not provide a consistent picture as to whether prosody, the melody and rhythm of speech, conveys a specific meaning. Perception studies show that English listeners assign meaning to prosodic patterns, and, for instance, associate some accents with contrast, whereas Dutch listeners behave more controversially. In two ERP studies we tested how Dutch listeners process words carrying two types of accents, which either provided new information (new information accents) or corrected information (corrective accents), both in single sentences (experiment 1) and after corrective and new information questions (experiment 2). In both experiments corrective accents elicited a sustained positivity as compared to new information accents, which started earlier in context than in single sentences. The positivity was not modulated by the nature of the preceding question, suggesting that the underlying neural mechanism likely reflects the construction of an interpretation to the accented word, either by identifying an alternative in context or by inferring it when no context is present. Our experimental results provide strong evidence for inferential processes related to prosodic contours in Dutch.     

The Spectral Diversity of Resting-State Fluctuations in the Human Brain

In order to assess whole-brain resting-state fluctuations at a wide range of frequencies, resting-state fMRI data of 20 healthy subjects were acquired using a multiband EPI sequence with a low TR (354 ms) and compared to 20 resting-state datasets from standard, high-TR (1800 ms) EPI scans. The spatial distribution of fluctuations in various frequency ranges are analyzed along with the spectra of the time-series in voxels from different regions of interest. Functional connectivity specific to different frequency ranges (<0.1 Hz; 0.1–0.25 Hz; 0.25–0.75 Hz; 0.75–1.4 Hz) was computed for both the low-TR and (for the two lower-frequency ranges) the high-TR datasets using bandpass filters. In the low-TR data, cortical regions exhibited highest contribution of low-frequency fluctuations and the most marked low-frequency peak in the spectrum, while the time courses in subcortical grey matter regions as well as the insula were strongly contaminated by high-frequency signals. White matter and CSF regions had highest contribution of high-frequency fluctuations and a mostly flat power spectrum. In the high-TR data, the basic patterns of the low-TR data can be recognized, but the high-frequency proportions of the signal fluctuations are folded into the low frequency range, thus obfuscating the low-frequency dynamics. Regions with higher proportion of high-frequency oscillations in the low-TR data showed flatter power spectra in the high-TR data due to aliasing of the high-frequency signal components, leading to loss of specificity in the signal from these regions in high-TR data. Functional connectivity analyses showed that there are correlations between resting-state signal fluctuations of distant brain regions even at high frequencies, which can be measured using low-TR fMRI. On the other hand, in the high-TR data, loss of specificity of measured fluctuations leads to lower sensitivity in detecting functional connectivity. This underlines the advantages of low-TR EPI sequences for resting-state and potentially also task-related fMRI experiments.     

Differential contributions of synaptic and intrinsic inhibitory currents to speech segmentation via flexible phase-locking in neural oscillators

Current hypotheses suggest that speech segmentation—the initial division and grouping of the speech stream into candidate phrases, syllables, and phonemes for further linguistic processing—is executed by a hierarchy of oscillators in auditory cortex. Theta (∼3-12 Hz) rhythms play a key role by phase-locking to recurring acoustic features marking syllable boundaries. Reliable synchronization to quasi-rhythmic inputs, whose variable frequency can dip below cortical theta frequencies (down to ∼1 Hz), requires “flexible” theta oscillators whose underlying neuronal mechanisms remain unknown. Using biophysical computational models, we found that the flexibility of phase-locking in neural oscillators depended on the types of hyperpolarizing currents that paced them. Simulated cortical theta oscillators flexibly phase-locked to slow inputs when these inputs caused both (i) spiking and (ii) the subsequent buildup of outward current sufficient to delay further spiking until the next input. The greatest flexibility in phase-locking arose from a synergistic interaction between intrinsic currents that was not replicated by synaptic currents at similar timescales. Flexibility in phase-locking enabled improved entrainment to speech input, optimal at mid-vocalic channels, which in turn supported syllabic-timescale segmentation through identification of vocalic nuclei. Our results suggest that synaptic and intrinsic inhibition contribute to frequency-restricted and -flexible phase-locking in neural oscillators, respectively. Their differential deployment may enable neural oscillators to play diverse roles, from reliable internal clocking to adaptive segmentation of quasi-regular sensory inputs like speech.     

Noise during rest enables the exploration of the brain's dynamic repertoire

Traditionally brain function is studied through measuring physiological responses in controlled sensory, motor, and cognitive paradigms. However, even at rest, in the absence of overt goal-directed behavior, collections of cortical regions consistently show temporally coherent activity. In humans, these resting state networks have been shown to greatly overlap with functional architectures present during consciously directed activity, which motivates the interpretation of rest activity as day dreaming, free association, stream of consciousness, and inner rehearsal. In monkeys, it has been shown though that similar coherent fluctuations are present during deep anesthesia when there is no consciousness. Here, we show that comparable resting state networks emerge from a stability analysis of the network dynamics using biologically realistic primate brain connectivity, although anatomical information alone does not identify the network. We specifically demonstrate that noise and time delays via propagation along connecting fibres are essential for the emergence of the coherent fluctuations of the default network. The spatiotemporal network dynamics evolves on multiple temporal scales and displays the intermittent neuroelectric oscillations in the fast frequency regimes, 1–100 Hz, commonly observed in electroencephalographic and magnetoencephalographic recordings, as well as the hemodynamic oscillations in the ultraslow regimes, <0.1 Hz, observed in functional magnetic resonance imaging. The combination of anatomical structure and time delays creates a space–time structure in which the neural noise enables the brain to explore various functional configurations representing its dynamic repertoire.     

Noisy Galvanic Vestibular Stimulation Modulates the Amplitude of EEG Synchrony Patterns

Noisy galvanic vestibular stimulation has been associated with numerous cognitive and behavioural effects, such as enhancement of visual memory in healthy individuals, improvement of visual deficits in stroke patients, as well as possibly improvement of motor function in Parkinson’s disease; yet, the mechanism of action is unclear. Since Parkinson’s and other neuropsychiatric diseases are characterized by maladaptive dynamics of brain rhythms, we investigated whether noisy galvanic vestibular stimulation was associated with measurable changes in EEG oscillatory rhythms within theta (4–7.5 Hz), low alpha (8–10 Hz), high alpha (10.5–12 Hz), beta (13–30 Hz) and gamma (31–50 Hz) bands. We recorded the EEG while simultaneously delivering noisy bilateral, bipolar stimulation at varying intensities of imperceptible currents – at 10, 26, 42, 58, 74 and 90% of sensory threshold – to ten neurologically healthy subjects. Using standard spectral analysis, we investigated the transient aftereffects of noisy stimulation on rhythms. Subsequently, using robust artifact rejection techniques and the Least Absolute Shrinkage Selection Operator regression and cross-validation, we assessed the combinations of channels and power spectral features within each EEG frequency band that were linearly related with stimulus intensity. We show that noisy galvanic vestibular stimulation predominantly leads to a mild suppression of gamma power in lateral regions immediately after stimulation, followed by delayed increase in beta and gamma power in frontal regions approximately 20–25 s after stimulation ceased. Ongoing changes in the power of each oscillatory band throughout frontal, central/parietal, occipital and bilateral electrodes predicted the intensity of galvanic vestibular stimulation in a stimulus-dependent manner, demonstrating linear effects of stimulation on brain rhythms. We propose that modulation of neural oscillations is a potential mechanism for the previously-described cognitive and motor effects of vestibular stimulation, and noisy galvanic vestibular stimulation may provide an additional non-invasive means for neuromodulation of functional brain networks.     

The Relation between Thematic Role Computing and Semantic Relatedness Processing during On-Line Sentence Comprehension

Sentence comprehension involves timely computing different types of relations between its verbs and noun arguments, such as morphosyntactic, semantic, and thematic relations. Here, we used EEG technique to investigate the potential differences in thematic role computing and lexical-semantic relatedness processing during on-line sentence comprehension, and the interaction between these two types of processes. Mandarin Chinese sentences were used as materials. The basic structure of those sentences is “Noun+Verb+‘le’+a two-character word”, with the Noun being the initial argument. The verb disambiguates the initial argument as an agent or a patient. Meanwhile, the initial argument and the verb are highly or lowly semantically related. The ERPs at the verbs revealed that: relative to the agent condition, the patient condition evoked a larger N400 only when the argument and verb were lowly semantically related; however, relative to the high-relatedness condition, the low-relatedness condition elicited a larger N400 regardless of the thematic relation; although both thematic role variation and semantic relatedness variation elicited N400 effects, the N400 effect elicited by the former was broadly distributed and reached maximum over the frontal electrodes, and the N400 effect elicited by the latter had a posterior distribution. In addition, the brain oscillations results showed that, although thematic role variation (patient vs. agent) induced power decreases around the beta frequency band (15–30 Hz), semantic relatedness variation (low-relatedness vs. high-relatedness) induced power increases in the theta frequency band (4–7 Hz). These results suggested that, in the sentence context, thematic role computing is modulated by the semantic relatedness between the verb and its argument; semantic relatedness processing, however, is in some degree independent from the thematic relations. Moreover, our results indicated that, during on-line sentence comprehension, thematic role computing and semantic relatedness processing are mediated by distinct neural systems.     

The Modulation Transfer Function for Speech Intelligibility

We systematically determined which spectrotemporal modulations in speech are necessary for comprehension by human listeners. Speech comprehension has been shown to be robust to spectral and temporal degradations, but the specific relevance of particular degradations is arguable due to the complexity of the joint spectral and temporal information in the speech signal. We applied a novel modulation filtering technique to recorded sentences to restrict acoustic information quantitatively and to obtain a joint spectrotemporal modulation transfer function for speech comprehension, the speech MTF. For American English, the speech MTF showed the criticality of low modulation frequencies in both time and frequency. Comprehension was significantly impaired when temporal modulations <12 Hz or spectral modulations <4 cycles/kHz were removed. More specifically, the MTF was bandpass in temporal modulations and low-pass in spectral modulations: temporal modulations from 1 to 7 Hz and spectral modulations <1 cycles/kHz were the most important. We evaluated the importance of spectrotemporal modulations for vocal gender identification and found a different region of interest: removing spectral modulations between 3 and 7 cycles/kHz significantly increases gender misidentifications of female speakers. The determination of the speech MTF furnishes an additional method for producing speech signals with reduced bandwidth but high intelligibility. Such compression could be used for audio applications such as file compression or noise removal and for clinical applications such as signal processing for cochlear implants.     

How Noise and Language Proficiency Influence Speech Recognition by Individual Non-Native Listeners

This study investigated how speech recognition in noise is affected by language proficiency for individual non-native speakers. The recognition of English and Chinese sentences was measured as a function of the signal-to-noise ratio (SNR) in sixty native Chinese speakers who never lived in an English-speaking environment. The recognition score for speech in quiet (which varied from 15%–92%) was found to be uncorrelated with speech recognition threshold (SRT_{Q /2}), i.e. the SNR at which the recognition score drops to 50% of the recognition score in quiet. This result demonstrates separable contributions of language proficiency and auditory processing to speech recognition in noise.     

Phase patterns of neuronal responses reliably discriminate speech in human auditory cortex

How natural speech is represented in the auditory cortex constitutes a major challenge for cognitive neuroscience. Although many single-unit and neuroimaging studies have yielded valuable insights about the processing of speech and matched complex sounds, the mechanisms underlying the analysis of speech dynamics in human auditory cortex remain largely unknown. Here, we show that the phase pattern of theta band (4-8 Hz) responses recorded from human auditory cortex with magnetoencephalography (MEG) reliably tracks and discriminates spoken sentences and that this discrimination ability is correlated with speech intelligibility. The findings suggest that an approximately 200 ms temporal window (period of theta oscillation) segments the incoming speech signal, resetting and sliding to track speech dynamics. This hypothesized mechanism for cortical speech analysis is based on the stimulus-induced modulation of inherent cortical rhythms and provides further evidence implicating the syllable as a computational primitive for the representation of spoken language.