Functional brain networks underlying perceptual switching: auditory streaming and verbal transformations

Recent studies have shown that auditory scene analysis involves distributed neural sites below, in, and beyond the auditory cortex (AC). However, it remains unclear what role each site plays and how they interact in the formation and selection of auditory percepts. We addressed this issue through perceptual multistability phenomena, namely, spontaneous perceptual switching in auditory streaming (AS) for a sequence of repeated triplet tones, and perceptual changes for a repeated word, known as verbal transformations (VTs). An event-related fMRI analysis revealed brain activity timelocked to perceptual switching in the cerebellum for AS, in frontal areas for VT, and the AC and thalamus for both. The results suggest that motor-based prediction, produced by neural networks outside the auditory system, plays essential roles in the segmentation of acoustic sequences both in AS and VT. The frequency of perceptual switching was determined by a balance between the activation of two sites, which are proposed to be involved in exploring novel perceptual organization and stabilizing current perceptual organization. The effect of the gene polymorphism of catechol-O-methyltransferase (COMT) on individual variations in switching frequency suggests that the balance of exploration and stabilization is modulated by catecholamines such as dopamine and noradrenalin. These mechanisms would support the noteworthy flexibility of auditory scene analysis.     

Perceptual organization of sound begins in the auditory periphery

Segmenting the complex acoustic mixture that makes a typical auditory scene into relevant perceptual objects is one of the main challenges of the auditory system [1], for both human and nonhuman species. Several recent studies indicate that perceptual auditory object formation, or "streaming," may be based on neural activity within the auditory cortex and beyond [2, 3]. Here, we find that scene analysis starts much earlier in the auditory pathways. Single units were recorded from a peripheral structure of the mammalian auditory brainstem, the cochlear nucleus. Peripheral responses were similar to cortical responses and displayed all of the functional properties required for streaming, including multisecond adaptation. Behavioral streaming was also measured in human listeners. Neurometric functions derived from the peripheral responses predicted accurately behavioral streaming. This reveals that subcortical structures may already contribute to the analysis of auditory scenes. This finding is consistent with the observation that species lacking a neocortex can still achieve and benefit from behavioral streaming [4]. For humans, we argue that auditory scene analysis of complex scenes is probably based on interactions between subcortical and cortical neural processes, with the relative contribution of each stage depending on the nature of the acoustic cues forming the streams.     

Perceptual Grouping over Time within and across Auditory and Tactile Modalities

In auditory scene analysis, population separation and temporal coherence have been proposed to explain how auditory features are grouped together and streamed over time. The present study investigated whether these two theories can be applied to tactile streaming and whether temporal coherence theory can be applied to crossmodal streaming. The results show that synchrony detection between two tones/taps at different frequencies/locations became difficult when one of the tones/taps was embedded in a perceptual stream. While the taps applied to the same location were streamed over time, the taps applied to different locations were not. This observation suggests that tactile stream formation can be explained by population-separation theory. On the other hand, temporally coherent auditory stimuli at different frequencies were streamed over time, but temporally coherent tactile stimuli applied to different locations were not. When there was within-modality streaming, temporally coherent auditory stimuli and tactile stimuli were not streamed over time, either. This observation suggests the limitation of temporal coherence theory when it is applied to perceptual grouping over time.     

Stimulus Phase Locking of Cortical Oscillation for Auditory Stream Segregation in Rats

The phase of cortical oscillations contains rich information and is valuable for encoding sound stimuli. Here we hypothesized that oscillatory phase modulation, instead of amplitude modulation, is a neural correlate of auditory streaming. Our behavioral evaluation provided compelling evidences for the first time that rats are able to organize auditory stream. Local field potentials (LFPs) were investigated in the cortical layer IV or deeper in the primary auditory cortex of anesthetized rats. In response to ABA- sequences with different inter-tone intervals and frequency differences, neurometric functions were characterized with phase locking as well as the band-specific amplitude evoked by test tones. Our results demonstrated that under large frequency differences and short inter-tone intervals, the neurometric function based on stimulus phase locking in higher frequency bands, particularly the gamma band, could better describe van Noorden’s perceptual boundary than the LFP amplitude. Furthermore, the gamma-band neurometric function showed a build-up-like effect within around 3 seconds from sequence onset. These findings suggest that phase locking and amplitude have different roles in neural computation, and support our hypothesis that temporal modulation of cortical oscillations should be considered to be neurophysiological mechanisms of auditory streaming, in addition to forward suppression, tonotopic separation, and multi-second adaptation.     

Modelling the Emergence and Dynamics of Perceptual Organisation in Auditory Streaming

Many sound sources can only be recognised from the pattern of sounds they emit, and not from the individual sound events that make up their emission sequences. Auditory scene analysis addresses the difficult task of interpreting the sound world in terms of an unknown number of discrete sound sources (causes) with possibly overlapping signals, and therefore of associating each event with the appropriate source. There are potentially many different ways in which incoming events can be assigned to different causes, which means that the auditory system has to choose between them. This problem has been studied for many years using the auditory streaming paradigm, and recently it has become apparent that instead of making one fixed perceptual decision, given sufficient time, auditory perception switches back and forth between the alternatives—a phenomenon known as perceptual bi- or multi-stability. We propose a new model of auditory scene analysis at the core of which is a process that seeks to discover predictable patterns in the ongoing sound sequence. Representations of predictable fragments are created on the fly, and are maintained, strengthened or weakened on the basis of their predictive success, and conflict with other representations. Auditory perceptual organisation emerges spontaneously from the nature of the competition between these representations. We present detailed comparisons between the model simulations and data from an auditory streaming experiment, and show that the model accounts for many important findings, including: the emergence of, and switching between, alternative organisations; the influence of stimulus parameters on perceptual dominance, switching rate and perceptual phase durations; and the build-up of auditory streaming. The principal contribution of the model is to show that a two-stage process of pattern discovery and competition between incompatible patterns can account for both the contents (perceptual organisations) and the dynamics of human perception in auditory streaming.     

Investigating the Neural Correlates of a Streaming Percept in an Informational-Masking Paradigm

Humans routinely segregate a complex acoustic scene into different auditory streams, through the extraction of bottom-up perceptual cues and the use of top-down selective attention. To determine the neural mechanisms underlying this process, neural responses obtained through magnetoencephalography (MEG) were correlated with behavioral performance in the context of an informational masking paradigm. In half the trials, subjects were asked to detect frequency deviants in a target stream, consisting of a rhythmic tone sequence, embedded in a separate masker stream composed of a random cloud of tones. In the other half of the trials, subjects were exposed to identical stimuli but asked to perform a different task—to detect tone-length changes in the random cloud of tones. In order to verify that the normalized neural response to the target sequence served as an indicator of streaming, we correlated neural responses with behavioral performance under a variety of stimulus parameters (target tone rate, target tone frequency, and the “protection zone”, that is, the spectral area with no tones around the target frequency) and attentional states (changing task objective while maintaining the same stimuli). In all conditions that facilitated target/masker streaming behaviorally, MEG normalized neural responses also changed in a manner consistent with the behavior. Thus, attending to the target stream caused a significant increase in power and phase coherence of the responses in recording channels correlated with an increase in the behavioral performance of the listeners. Normalized neural target responses also increased as the protection zone widened and as the frequency of the target tones increased. Finally, when the target sequence rate increased, the buildup of the normalized neural responses was significantly faster, mirroring the accelerated buildup of the streaming percepts. Our data thus support close links between the perceptual and neural consequences of the auditory stream segregation.     

Object selection by an oscillatory neural network

We describe a new solution to the problem of consecutive selection of objects in a visual scene by an oscillatory neural network with the global interaction realised through a central executive element (central oscillator). The frequency coding is used to represent greyscale images in the network. The functioning of the network is based on three main principles: (1) the synchronisation of oscillators via phase-locking, (2) adaptation of the natural frequency of the central oscillator, and (3) resonant increase of the amplitudes of the oscillators which work in-phase with the central oscillator. Examples of network simulations are presented to show the reliability of the results of consecutive selection of objects under conditions of constant and varying brightness of the objects.     

Frequency Change Detection in Human Auditory Cortex

We offer a model of how human cortex detects changes in the auditory environment. Auditory change detection has recently been the object of intense investigation via the mismatch negativity (MMN). MMN is a preattentive response to sudden changes in stimulation, measured noninvasively in the electroencephalogram (EEG) and the magnetoencephalogram (MEG). It is elicited in the oddball paradigm, where infrequent deviant tones intersperse a series of repetitive standard tones. However, little apart from the participation of tonotopically organized auditory cortex is known about the neural mechanisms underlying change detection and the MMN. In the present study, we investigate how poststimulus inhibition might account for MMN and compare the effects of adaptation with those of lateral inhibition in a model describing tonotopically organized cortex. To test the predictions of our model, we performed MEG and EEG measurements on human subjects and used both small- (<1/3 octave) and large- (>5 octaves) frequency differences between the standard and deviant tones. The experimental results bear out the prediction that MMN is due to both adaptation and lateral inhibition. Finally, we suggest that MMN might serve as a probe of what stimulus features are mapped by human auditory cortex.     

Auditory cortical contrast enhancing by global winner-take-all inhibitory interactions

Brains decompose the world into discrete objects of perception, thereby facing the problem of how to segregate and selectively address similar objects that are concurrently present in a scene. Theoretical models propose that this could be achieved by neuronal implementations of so-called winner-take-all algorithms where neuronal representations of objects or object features interact in a competitive manner. Here we present evidence for the existence of such a mechanism in an animal species. We present electrophysiological, neuropharmacological and neuroanatomical data which suggest a novel view of the role of GABA(A)-mediated inhibition in primary auditory cortex (AI), where intracortical GABA(A)-mediated inhibition operates on a global scale within a circular map of sound periodicity representation in AI, with functionally inhibitory projections of similar effect from any location throughout the whole map. These interactions could underlie the proposed competitive "winner-take-all" algorithm to support object segregation, e.g., segregation of different speakers in cocktail-party situations.     

Neural adaptation facilitates oscillatory responses to static inputs in a recurrent network of ON and OFF cells

We investigate the role of adaptation in a neural field model, composed of ON and OFF cells, with delayed all-to-all recurrent connections. As external spatially profiled inputs drive the network, ON cells receive inputs directly, while OFF cells receive an inverted image of the original signals. Via global and delayed inhibitory connections, these signals can cause the system to enter states of sustained oscillatory activity. We perform a bifurcation analysis of our model to elucidate how neural adaptation influences the ability of the network to exhibit oscillatory activity. We show that slow adaptation encourages input-induced rhythmic states by decreasing the Andronov–Hopf bifurcation threshold. We further determine how the feedback and adaptation together shape the resonant properties of the ON and OFF cell network and how this affects the response to time-periodic input. By introducing an additional frequency in the system, adaptation alters the resonance frequency by shifting the peaks where the response is maximal. We support these results with numerical experiments of the neural field model. Although developed in the context of the circuitry of the electric sense, these results are applicable to any network of spontaneously firing cells with global inhibitory feedback to themselves, in which a fraction of these cells receive external input directly, while the remaining ones receive an inverted version of this input via feedforward di-synaptic inhibition. Thus the results are relevant beyond the many sensory systems where ON and OFF cells are usually identified, and provide the backbone for understanding dynamical network effects of lateral connections and various forms of ON/OFF responses.     

Fast and robust image segmentation by small-world neural oscillator networks

Inspired by the temporal correlation theory of brain functions, researchers have presented a number of neural oscillator networks to implement visual scene segmentation problems. Recently, it is shown that many biological neural networks are typical small-world networks. In this paper, we propose and investigate two small-world models derived from the well-known LEGION (locally excitatory and globally inhibitory oscillator network) model. To form a small-world network, we add a proper proportion of unidirectional shortcuts (random long-range connections) to the original LEGION model. With local connections and shortcuts, the neural oscillators can not only communicate with neighbors but also exchange phase information with remote partners. Model 1 introduces excitatory shortcuts to enhance the synchronization within an oscillator group representing the same object. Model 2 goes further to replace the global inhibitor with a sparse set of inhibitory shortcuts. Simulation results indicate that the proposed small-world models could achieve synchronization faster than the original LEGION model and are more likely to bind disconnected image regions belonging together. In addition, we argue that these two models are more biologically plausible.     

Sound localization: Jeffress and beyond

Many animals use the interaural time differences (ITDs) to locate the source of low frequency sounds. The place coding theory proposed by Jeffress has long been a dominant model to account for the neural mechanisms of ITD detection. Recent research, however, suggests a wider range of strategies for ITD coding in the binaural auditory brainstem. We discuss how ITD is coded in avian, mammalian, and reptilian nervous systems, and review underlying synaptic and cellular properties that enable precise temporal computation. The latest advances in recording and analysis techniques provide powerful tools for both overcoming and utilizing the large field potentials in these nuclei.     

Differential contributions of synaptic and intrinsic inhibitory currents to speech segmentation via flexible phase-locking in neural oscillators

Current hypotheses suggest that speech segmentation—the initial division and grouping of the speech stream into candidate phrases, syllables, and phonemes for further linguistic processing—is executed by a hierarchy of oscillators in auditory cortex. Theta (∼3-12 Hz) rhythms play a key role by phase-locking to recurring acoustic features marking syllable boundaries. Reliable synchronization to quasi-rhythmic inputs, whose variable frequency can dip below cortical theta frequencies (down to ∼1 Hz), requires “flexible” theta oscillators whose underlying neuronal mechanisms remain unknown. Using biophysical computational models, we found that the flexibility of phase-locking in neural oscillators depended on the types of hyperpolarizing currents that paced them. Simulated cortical theta oscillators flexibly phase-locked to slow inputs when these inputs caused both (i) spiking and (ii) the subsequent buildup of outward current sufficient to delay further spiking until the next input. The greatest flexibility in phase-locking arose from a synergistic interaction between intrinsic currents that was not replicated by synaptic currents at similar timescales. Flexibility in phase-locking enabled improved entrainment to speech input, optimal at mid-vocalic channels, which in turn supported syllabic-timescale segmentation through identification of vocalic nuclei. Our results suggest that synaptic and intrinsic inhibition contribute to frequency-restricted and -flexible phase-locking in neural oscillators, respectively. Their differential deployment may enable neural oscillators to play diverse roles, from reliable internal clocking to adaptive segmentation of quasi-regular sensory inputs like speech.     

Temporal coding in the guinea-pig auditory cortex as revealed by optical imaging and its pattern-time-series analysis

The neural network structure of a guinea-pig's primary auditory cortex is estimated by applying pattern-time-series analysis to the auditory evoked responses. Spatiotemporal patterns in click-evoked responses, observed by optical recording with voltage-sensitive dye, are analyzed by time series analysis using a multivariable autoregressive (MAR) model. Oscillatory neural activities with a distribution of about 10 40 Hz in the click-induced evoked responses are found in the cortical response field. The cortical regions where the distributed neural oscillations are generated are identified by pattern-time-series analysis. In addition, two types of cortico-cortical connections, unilateral and bilateral connections between the cortical points, are speculated to be the causes of oscillatory neural activity transfer. It can be said that the so-called synchronized neural oscillation, in the sense of coherency or correlation between the two evoked responses at the oscillatory frequency, does not necessarily represent real corticocortical neural connections at the evoked response points.     

An oscillator network exhibiting a long-lasting response to an external signal

This study proposes an oscillator network to model the long-lasting responses observed in neural circuits. The responses of the proposed network model are represented by the temporal synchronization of the oscillators. The response duration does not depend on the natural frequency of the oscillators, which allows the responses to last much longer than the oscillation period of the oscillators. We can control the response duration by tuning the connection strengths between the oscillators and the external signal that triggers the responses. It is possible to break and restart the responses regardless of the way in which the oscillators are connected.     

Interaction between Attention and Bottom-Up Saliency Mediates the Representation of Foreground and Background in an Auditory Scene

The mechanism by which a complex auditory scene is parsed into coherent objects depends on poorly understood interactions between task-driven and stimulus-driven attentional processes. We illuminate these interactions in a simultaneous behavioral–neurophysiological study in which we manipulate participants' attention to different features of an auditory scene (with a regular target embedded in an irregular background). Our experimental results reveal that attention to the target, rather than to the background, correlates with a sustained (steady-state) increase in the measured neural target representation over the entire stimulus sequence, beyond auditory attention's well-known transient effects on onset responses. This enhancement, in both power and phase coherence, occurs exclusively at the frequency of the target rhythm, and is only revealed when contrasting two attentional states that direct participants' focus to different features of the acoustic stimulus. The enhancement originates in auditory cortex and covaries with both behavioral task and the bottom-up saliency of the target. Furthermore, the target's perceptual detectability improves over time, correlating strongly, within participants, with the target representation's neural buildup. These results have substantial implications for models of foreground/background organization, supporting a role of neuronal temporal synchrony in mediating auditory object formation.     

Dynamic neural field with local inhibition

A lateral-inhibition type neural field model with restricted connections is presented here and represents an experimental extension of the continuum neural field theory (CNFT) by suppression of the global inhibition. A modified CNFT equation is introduced and allows for a locally defined inhibition to spatially expand within the network and results in a global competition extending far beyond the range of local connections by virtue of diffusion of inhibition. The resulting model is able to attend to a moving stimulus in the presence of a very high level of noise, several distractors or a mixture of both.     

Control of interaction strength in a network of the true slime mold by a microfabricated structure

The plasmodium of the true slime mold, Physarum polycephalum, which shows various nonlinear oscillatory phenomena, for example, in its thickness, protoplasmic streaming and concentration of intracellular chemicals, can be regarded as a collective of nonlinear oscillators. The plasmodial oscillators are interconnected by microscale tubes whose dimensions can be closely related to the strength of interaction between the oscillators. Investigation of the collective behavior of the oscillators under the conditions in which the interaction strength can be systematically controlled gives significant information on the characteristics of the system. In this study, we proposed a living model system of a coupled oscillator system in the Physarum plasmodium. We patterned the geometry and dimensions of the microscale tube structure in the plasmodium by a microfabricated structure (microstructure). As the first step, we constructed a two-oscillator system for the plasmodium that has two wells (oscillator part) and a channel (coupling part). We investigated the oscillation behavior by monitoring the thickness oscillation of the plasmodium in the microstructure with various channel widths. It was found that the oscillation behavior of two oscillators dynamically changed depending on the channel width. Based on the results of measurements of the tube dimensions and the velocity of the protoplasmic streaming in the tube, we discuss how the channel width relates to the interaction strength of the coupled oscillator system.     

Reverberation challenges the temporal representation of the pitch of complex sounds

Accurate neural coding of the pitch of complex sounds is an essential part of auditory scene analysis; differences in pitch help segregate concurrent sounds, while similarities in pitch can help group sounds from a common source. In quiet, nonreverberant backgrounds, pitch can be derived from timing information in broadband high-frequency auditory channels and/or from frequency and timing information carried in narrowband low-frequency auditory channels. Recording from single neurons in the cochlear nucleus of anesthetized guinea pigs, we show that the neural representation of pitch based on timing information is severely degraded in the presence of reverberation. This degradation increases with both increasing reverberation strength and channel bandwidth. In a parallel human psychophysical pitch-discrimination task, reverberation impaired the ability to distinguish a high-pass harmonic sound from noise. Together, these findings explain the origin of perceptual difficulties experienced by both normal-hearing and hearing-impaired listeners in reverberant spaces.