Acoustic versus phonetic representation of speech as reflected by the mismatch negativity event-related potential

The concept of categorical perception of speech and speech-like sounds has been central to models of speech perception for decades. Event-related potentials (ERPs) provide a neurophysiologic perspective of this important phenomenon. In the present experiment the mismatch negativity (MMN) event-related potential, which is sensitive to fine acoustic differences, was recorded in adults. Of interest was whether the MMN reflects the acoustic or categorical perception of speech.The MMN was elicited by stimulus pairs (along a continuum varying in place of articulation from /da/ to /ga/) which had been identified as the same phoneme /da/ (within category condition) and as different phonemes /da/ and /ga/ (across categories condition). The acoustic differences between these two pairs of stimuli were equivalent.The MMN was observed in all subjects both in the within and across category conditions. Furthermore, the MMN did not differ in latency, amplitude or area within and across categories. That is, the MMN indicated equal discrimination both across and within categories. These results suggest that the MMN appears to reflect the processing of acoustic aspects of the speech stimulus, but not phonetic processing into categories. The MMN appears to be an extremely sensitive electrophysiologic index of minimal acoustic differences in speech stimuli.     

Sequential grouping of tone sequence as reflected by the mismatch negativity

The human sequential grouping that organizes parts of tones into a group was examined by the mismatch negativity (MMN), a component of event-related potentials that reveals the sensory memory process. The sequential grouping is accomplished by the combinations of some factors, e.g., temporal and frequency proximity principles. In this study, auditory oddball stimuli in which each of the stimuli consisted of series of tone bursts, were applied to the subjects, and the MMN elicited by the deviation of the frequency of the last tone in the stimulus was investigated. The relationship between the expected phenomena of sequential grouping of tones and observed magnitudes of MMN was evaluated. It was shown that the magnitudes of MMN changed according to the configuration (number of tones, frequency) of tone sequence to be stored. This result suggested that the sequential grouping of presented tones was achieved on the preattentive auditory sensory memory process. It was also shown that the relative change of MMN magnitudes corresponded to the conditions of sequential grouping, which had been proposed by the auditory psychophysical studies. The investigation of MMN properties could reveal the nature of auditory sequential grouping.This study was approved by the Ethics Committee on Clinical Investigation, Graduate School of Engineering, Tohoku University and was carried out in accordance with the policy of the Declaration of Helsinki.     

Short-term replicability of the mismatch negativity

The short-term replicability of the mismatch negativity (MMN) between two recording sessions spaced 2 h apart was evaluated at individual and group levels in a sample of 11 healthy adults. Subjects were presented with a random sequence of 1000 Hz standard (92%) and 1100 Hz deviant (8%) tones while they were reading a book. The N1 and P2 exogenous components to standard tones showed a fairly good individual and group replicability. There were no significant differences in the MMN amplitude and latency between the two sessions in the group of subjects as a whole. The individual replicability of the MMN was not as good as for the N1 to standards, reaching significance in only some of the electrodes. This result was, however, similar to that obtained for the N1 after deviant tones. The results indicate that the MMN has good replicability at the group level, and further that at the individual level, MMN replicability is similar to that of the N1 to deviants. This suggests that the number of summations should be increased in order to improve the clinical usefulness of the MMN.     

The self-organization of speech sounds

The speech code is a vehicle of language: it defines a set of forms used by a community to carry information. Such a code is necessary to support the linguistic interactions that allow humans to communicate. How then may a speech code be formed prior to the existence of linguistic interactions? Moreover, the human speech code is discrete and compositional, shared by all the individuals of a community but different across communities, and phoneme inventories are characterized by statistical regularities. How can a speech code with these properties form? We try to approach these questions in the paper, using the "methodology of the artificial". We build a society of artificial agents, and detail a mechanism that shows the formation of a discrete speech code without pre-supposing the existence of linguistic capacities or of coordinated interactions. The mechanism is based on a low-level model of sensory-motor interactions. We show that the integration of certain very simple and non-language-specific neural devices leads to the formation of a speech code that has properties similar to the human speech code. This result relies on the self-organizing properties of a generic coupling between perception and production within agents, and on the interactions between agents. The artificial system helps us to develop better intuitions on how speech might have appeared, by showing how self-organization might have helped natural selection to find speech.     

Variability and replicability of the mismatch negativity

The interindividual variation and test-retest stability of the mismatch negativity (MMN) and N1 components of the event-related potential (ERP) were investigated by presenting standard (85%) and deviant tones (15%) to 10 young subjects in 2 sessions separated by 1 month. Deviant tones in different blocks were either frequency or duration changes with interstimulus intervals (ISIS) of 0.5 and 1.5 sec. The results showed a fairly good test-retest stability of the MMN amplitude for both types of changes with each IS[ at the group level. The amplitude of the duration MMN showed significant individual test-retest stability. The N1 amplitude showed high stability at both the group and individual levels. Both the MMN and N1 showed considerable interindividual variation. The results suggest that MMN and N1 can be used in follow-up studies not only at the group level but possibly at the individual level also.     

Integration of consonant and pitch processing as revealed by the absence of additivity in mismatch negativity

Consonants, unlike vowels, are thought to be speech specific and therefore no interactions would be expected between consonants and pitch, a basic element for musical tones. The present study used an electrophysiological approach to investigate whether, contrary to this view, there is integrative processing of consonants and pitch by measuring additivity of changes in the mismatch negativity (MMN) of evoked potentials. The MMN is elicited by discriminable variations occurring in a sequence of repetitive, homogeneous sounds. In the experiment, event-related potentials (ERPs) were recorded while participants heard frequently sung consonant-vowel syllables and rare stimuli deviating in either consonant identity only, pitch only, or in both dimensions. Every type of deviation elicited a reliable MMN. As expected, the two single-deviant MMNs had similar amplitudes, but that of the double-deviant MMN was also not significantly different from them. This absence of additivity in the double-deviant MMN suggests that consonant and pitch variations are processed, at least at a pre-attentive level, in an integrated rather than independent way. Domain-specificity of consonants may depend on higher-level processes in the hierarchy of speech perception.     

Auditory stimulus discrimination recorded in dogs, as indicated by mismatch negativity (MMN)

Dog cognition research tends to rely on behavioural response, which can be confounded by obedience or motivation, as the primary means of indexing dog cognitive abilities. A physiological method of measuring dog cognitive processing would be instructive and could complement behavioural response. Electroencephalogram (EEG) has been used in humans to study stimulus processing, which results in waveforms called event-related potentials (ERPs). One ERP component, mismatch negativity (MMN), is a negative deflection approximately 160-200 ms after stimulus onset, which may be related to change detection from echoic sensory memory. We adapted a minimally invasive technique to record MMN in dogs. Dogs were exposed to an auditory oddball paradigm in which deviant tones (10% probability) were pseudo-randomly interspersed throughout an 8 min sequence of standard tones (90% probability). A significant difference in MMN ERP amplitude was observed after the deviant tone in comparison to the standard tone, t₅ = −2.98, p = 0.03. This difference, attributed to discrimination of an unexpected stimulus in a series of expected stimuli, was not observed when both tones occurred 50% of the time, t₁ = −0.82, p > 0.05. Dogs showed no evidence of pain or distress at any point. We believe this is the first illustration of MMN in a group of dogs and anticipate that this technique may provide valuable insights in cognitive tasks such as object discrimination.     

Preattentive extraction of abstract auditory rules in speech sound stream: a mismatch negativity study using lexical tones

Background

Extraction of linguistically relevant auditory features is critical for speech comprehension in complex auditory environments, in which the relationships between acoustic stimuli are often abstract and constant while the stimuli per se are varying. These relationships are referred to as the abstract auditory rule in speech and have been investigated for their underlying neural mechanisms at an attentive stage. However, the issue of whether or not there is a sensory intelligence that enables one to automatically encode abstract auditory rules in speech at a preattentive stage has not yet been thoroughly addressed.

Methodology/Principal Findings

We chose Chinese lexical tones for the current study because they help to define word meaning and hence facilitate the fabrication of an abstract auditory rule in a speech sound stream. We continuously presented native Chinese speakers with Chinese vowels differing in formant, intensity, and level of pitch to construct a complex and varying auditory stream. In this stream, most of the sounds shared flat lexical tones to form an embedded abstract auditory rule. Occasionally the rule was randomly violated by those with a rising or falling lexical tone. The results showed that the violation of the abstract auditory rule of lexical tones evoked a robust preattentive auditory response, as revealed by whole-head electrical recordings of the mismatch negativity (MMN), though none of the subjects acquired explicit knowledge of the rule or became aware of the violation.

Conclusions/Significance

Our results demonstrate that there is an auditory sensory intelligence in the perception of Chinese lexical tones. The existence of this intelligence suggests that the humans can automatically extract abstract auditory rules in speech at a preattentive stage to ensure speech communication in complex and noisy auditory environments without drawing on conscious resources.     

Allocentric or craniocentric representation of acoustic space: an electrotomography study using mismatch negativity

The world around us appears stable in spite of our constantly moving head, eyes, and body. How this is achieved by our brain is hardly understood and even less so in the auditory domain. Using electroencephalography and the so-called mismatch negativity, we investigated whether auditory space is encoded in an allocentric (referenced to the environment) or craniocentric representation (referenced to the head). Fourteen subjects were presented with noise bursts from loudspeakers in an anechoic environment. Occasionally, subjects were cued to rotate their heads and a deviant sound burst occurred, that deviated from the preceding standard stimulus either in terms of an allocentric or craniocentric frame of reference. We observed a significant mismatch negativity, i.e., a more negative response to deviants with reference to standard stimuli from about 136 to 188 ms after stimulus onset in the craniocentric deviant condition only. Distributed source modeling with sLORETA revealed an involvement of lateral superior temporal gyrus and inferior parietal lobule in the underlying neural processes. These findings suggested a craniocentric, rather than allocentric, representation of auditory space at the level of the mismatch negativity.     

The mismatch negativity for duration decrement of auditory stimuli in healthy subjects

The amplitude and latency of the mismatch negativity (MMN) elicited by occasional shorter-duration tones (25 and 50 ms) in a sequence of 75 ms standard tones were studied in 40 healthy subjects (9–84 years). The replicability and age dependence of the MMN-responses were determined. The 25 ms deviant tone evoked a clear response in 39 of the subjects, while the 50 ms deviant tone evoked an observable MMN only in 32 of the subjects. The MMN peak amplitude for the 25 ms deviants was significantly larger than for the 50 ms deviants. There was no significant difference in the peak latencies (measured from stimulus offset). For the 25 ms deviant, the amplitude diminished with increasing age. The MMN curves for the 25 ms deviant, measured on separate days in 14 subjects, looked very replicable. As a result of noise and filtering effect, the product-moment correlations were poor. The results indicate that the signal-to-noise ratio for the MMN to 25 ms deviants, obtained even in a 25 min recording session, is large enough for clinical use and individual diagnostics when undetectable (or very low amplitude) MMN is used as a sign of pathology. However, judged from the low correlation coefficients, despite the good replicability in visual evaluation, better methods for MMN quantification have to be used for clinical follow-up.     

Zebra finches exhibit speaker-independent phonetic perception of human speech

Humans readily distinguish spoken words that closely resemble each other in acoustic structure, irrespective of audible differences between individual voices or sex of the speakers. There is an ongoing debate about whether the ability to form phonetic categories that underlie such distinctions indicates the presence of uniquely evolved, speech-linked perceptual abilities, or is based on more general ones shared with other species. We demonstrate that zebra finches (Taeniopygia guttata) can discriminate and categorize monosyllabic words that differ in their vowel and transfer this categorization to the same words spoken by novel speakers independent of the sex of the voices. Our analysis indicates that the birds, like humans, use intrinsic and extrinsic speaker normalization to make the categorization. This finding shows that there is no need to invoke special mechanisms, evolved together with language, to explain this feature of speech perception.     

Differentiation of schizophrenia patients from healthy subjects by mismatch negativity and neuropsychological tests

Background

Schizophrenia is a heterogeneous disorder with diverse presentations. The current and the proposed DSM-V diagnostic system remains phenomenologically based, despite the fact that several neurobiological and neuropsychological markers have been identified. A multivariate approach has better diagnostic utility than a single marker method. In this study, the mismatch negativity (MMN) deficit of schizophrenia was first replicated in a Han Chinese population, and then the MMN was combined with several neuropsychological measurements to differentiate schizophrenia patients from healthy subjects.

Methodology/Principal Findings

120 schizophrenia patients and 76 healthy controls were recruited. Each subject received examinations for duration MMN, Continuous Performance Test, Wisconsin Card Sorting Test, and Wechsler Adult Intelligence Scale Third Edition (WAIS-III). The MMN was compared between cases and controls, and important covariates were investigated. Schizophrenia patients had significantly reduced MMN amplitudes, and MMN decreased with increasing age in both patient and control groups. None of the neuropsychological indices correlated with MMN. Predictive multivariate logistic regression models using the MMN and neuropsychological measurements as predictors were developed. Four predictors, including MMN at electrode FCz and three scores from the WAIS-III (Arithmetic, Block Design, and Performance IQ) were retained in the final predictive model. The model performed well in differentiating patients from healthy subjects (percentage of concordant pairs: 90.5%).

Conclusions/Significance

MMN deficits were found in Han Chinese schizophrenia patients. The multivariate approach combining biomarkers from different modalities such as electrophysiology and neuropsychology had a better diagnostic utility.     

The habituation of event-related potentials to speech sounds and tones

We examined the short- and long-term habituation of auditory event-related potentials (ERPs) elicited by tones, complex tones and digitized speech sounds (vowels and consonant-vowel-consonant syllables). Twelve different stimuli equated in loudness and duration (300 msec) were studied. To examine short-term habituation stimuli were presented in trains of 6 with interstimulus intervals of 0.5 or 1.0 sec. The first 4 stimuli in a train were identical standards. On 50% of the trains the standard in the 5th position was replaced by a deviant probe stimulus, and on 20% of the trains the standard in the 6th position was replaced by a target, a truncated standard that required a speeded button press response.Short-term habituation (STH) was complete by the third stimulus in the train and resulted in amplitude decrements of 50–75% for the N1 component. STH was partially stimulus specific in that amplitudes were larger following deviant stimuli in the 5th position than following standards. STH of the N1 was more marked for speech sounds than for loudness-matched tones or complex tones at short ISI. In addition, standard and deviant stimuli that differed in phonetic structure showed more cross-habituation than did tones or complex tones that differed in frequency. This pattern of results suggests that STH is a function of the acoustic resemblance of successive stimuli.The long-term habituation (LTH) of the ERP was studied by comparing amplitudes across balanced 5.25 m stimulus blocks over the course of the experiment. Two types of LTH were observed. The N1 showed stimulus-specific LTH in that N1 amplitudes declined during the presentation of a stimulus, but returned to control levels when a different stimulus was presented in the subsequent condition. In contrast, the P3 elicited by the deviant stimuli showed non-specific LTH, being reduced across successive blocks containing different stimuli. P3s elicited by target stimuli remained stable in amplitude.     

Auditory perception: hearing the texture of sounds

A recent study provides intriguing insights into how we recognize the sound of everyday objects from the statistical properties of the textures they produce.     

Listening to speech in the presence of other sounds

Although most research on the perception of speech has been conducted with speech presented without any competing sounds, we almost always listen to speech against a background of other sounds which we are adept at ignoring. Nevertheless, such additional irrelevant sounds can cause severe problems for speech recognition algorithms and for the hard of hearing as well as posing a challenge to theories of speech perception. A variety of different problems are created by the presence of additional sound sources: detection of features that are partially masked, allocation of detected features to the appropriate sound sources and recognition of sounds on the basis of partial information. The separation of sounds is arousing substantial attention in psychoacoustics and in computer science. An effective solution to the problem of separating sounds would have important practical applications.