Processing of complex stimuli and natural scenes in the auditory cortex

Neuronal responses in auditory cortex show a fascinating mixture of characteristics that span the range from almost perfect copies of physical aspects of the stimuli to extremely complex context-dependent responses. Fast, highly stimulus-specific adaptation and slower plastic mechanisms work together to constantly adjust neuronal response properties to the statistics of the auditory scene. Evidence with converging implications suggests that the neuronal activity in primary auditory cortex represents sounds in terms of auditory objects rather than in terms of invariant acoustic features.     

Probing the cortical neuronal correlates of a sensory discrimination process

Key to understanding perception is the form of how sensory stimuli are represented in the evoked activity of the brain. Here, we addressed the question of which components of the evoked neuronal activity in the somatosensory cortex represent the stimulus features while trained monkeys discriminated the difference in frequency between two vibrotactile stimuli. We probed whether these cortical neuronal representations are essential to perception. The results show a strong link between the cortical representation of the stimulus and perception.     

The auditory cortex

The division of the auditory cortex into various fields, functional aspects of these fields, and neuronal coding in the primary auditory cortical field (AI) are reviewed with stress on features that may be common to mammals. On the basis of 14 topographies and clustered distributions of neuronal response characteristics in the primary auditory cortical field, a hypothesis is developed of how a certain complex acoustic pattern may be encoded in an equivalent spatial activity pattern in AI, generated by time-coordinated firing of groups of neurons. The auditory cortex, demonstrated specifically for AI, appears to perform sound analysis by synthesis, i.e. by combining spatially distributed coincident or time-coordinated neuronal responses. The dynamics of sounds and the plasticity of cortical responses are considered as a topic for research. Accepted: 25 July 1997     

Deep Supervised,but Not Unsupervised,Models May Explain IT Cortical Representation

Inferior temporal (IT) cortex in human and nonhuman primates serves visual object recognition. Computational object-vision models, although continually improving, do not yet reach human performance. It is unclear to what extent the internal representations of computational models can explain the IT representation. Here we investigate a wide range of computational model representations (37 in total), testing their categorization performance and their ability to account for the IT representational geometry. The models include well-known neuroscientific object-recognition models (e.g. HMAX, VisNet) along with several models from computer vision (e.g. SIFT, GIST, self-similarity features, and a deep convolutional neural network). We compared the representational dissimilarity matrices (RDMs) of the model representations with the RDMs obtained from human IT (measured with fMRI) and monkey IT (measured with cell recording) for the same set of stimuli (not used in training the models). Better performing models were more similar to IT in that they showed greater clustering of representational patterns by category. In addition, better performing models also more strongly resembled IT in terms of their within-category representational dissimilarities. Representational geometries were significantly correlated between IT and many of the models. However, the categorical clustering observed in IT was largely unexplained by the unsupervised models. The deep convolutional network, which was trained by supervision with over a million category-labeled images, reached the highest categorization performance and also best explained IT, although it did not fully explain the IT data. Combining the features of this model with appropriate weights and adding linear combinations that maximize the margin between animate and inanimate objects and between faces and other objects yielded a representation that fully explained our IT data. Overall, our results suggest that explaining IT requires computational features trained through supervised learning to emphasize the behaviorally important categorical divisions prominently reflected in IT.     

Neuronal Categorization and Discrimination of Social Behaviors in Primate Prefrontal Cortex

It has been implied that primates have an ability to categorize social behaviors between other individuals for the execution of adequate social-interactions. Since the lateral prefrontal cortex (LPFC) is involved in both the categorization and the processing of social information, the primate LPFC may be involved in the categorization of social behaviors. To test this hypothesis, we examined neuronal activity in the LPFC of monkeys during presentations of two types of movies of social behaviors (grooming, mounting) and movies of plural monkeys without any eye- or body-contacts between them (no-contacts movies). Although the monkeys were not required to categorize and discriminate the movies in this task, a subset of neurons sampled from the LPFC showed a significantly different activity during the presentation of a specific type of social behaviors in comparison with the others. These neurons categorized social behaviors at the population level and, at the individual neuron level, the majority of the neurons discriminated each movie within the same category of social behaviors. Our findings suggest that a fraction of LPFC neurons process categorical and discriminative information of social behaviors, thereby contributing to the adaptation to social environments.     

Preattentive extraction of abstract auditory rules in speech sound stream: a mismatch negativity study using lexical tones

Background

Extraction of linguistically relevant auditory features is critical for speech comprehension in complex auditory environments, in which the relationships between acoustic stimuli are often abstract and constant while the stimuli per se are varying. These relationships are referred to as the abstract auditory rule in speech and have been investigated for their underlying neural mechanisms at an attentive stage. However, the issue of whether or not there is a sensory intelligence that enables one to automatically encode abstract auditory rules in speech at a preattentive stage has not yet been thoroughly addressed.

Methodology/Principal Findings

We chose Chinese lexical tones for the current study because they help to define word meaning and hence facilitate the fabrication of an abstract auditory rule in a speech sound stream. We continuously presented native Chinese speakers with Chinese vowels differing in formant, intensity, and level of pitch to construct a complex and varying auditory stream. In this stream, most of the sounds shared flat lexical tones to form an embedded abstract auditory rule. Occasionally the rule was randomly violated by those with a rising or falling lexical tone. The results showed that the violation of the abstract auditory rule of lexical tones evoked a robust preattentive auditory response, as revealed by whole-head electrical recordings of the mismatch negativity (MMN), though none of the subjects acquired explicit knowledge of the rule or became aware of the violation.

Conclusions/Significance

Our results demonstrate that there is an auditory sensory intelligence in the perception of Chinese lexical tones. The existence of this intelligence suggests that the humans can automatically extract abstract auditory rules in speech at a preattentive stage to ensure speech communication in complex and noisy auditory environments without drawing on conscious resources.     

Specificity of sensorimotor learning and the neural code: Neuronal representations in the primary motor cortex

Human studies show that the learning of a new sensorimotor mapping that requires adaptation to directional errors is local and generalizes poorly to untrained directions. We trained monkeys to learn new visuomotor rotations for only one target in space and recorded neuronal activity in the primary motor cortex before, during and after learning. Similar to humans, the monkeys showed poor transfer of learning to other directions, as observed by behavioral aftereffects for untrained directions. To test for internal representations underlying these changes, we compared two features of neuronal activity before and after learning: changes in firing rates and changes in information content. Specific elevations of firing rate were only observed in a subpopulation of cells in the motor cortex with directional properties corresponding to the locally learned rotation; namely cells only showed plasticity if their preferred direction was near the training one. We applied measures from information theory to probe for learning-related changes in the neuronal code. Single cells conveyed more information about the direction of movement and this specific improvement in encoding was correlated with an increase in the slope of the neurons' tuning curve. Further, the improved information after learning enabled a more accurate reconstruction of movement direction from neuronal populations. Our findings suggest a neural mechanism for the confined generalization of a newly acquired internal model by showing a tight relationship between the locality of learning and the properties of neurons. They also provide direct evidence for improvement in the neural code as a result of learning.     

Defining the V5/MT neuronal pool for perceptual decisions in a visual stereo-motion task

In the primate visual cortex, neurons signal differences in the appearance of objects with high precision. However, not all activated neurons contribute directly to perception. We defined the perceptual pool in extrastriate visual area V5/MT for a stereo-motion task, based on trial-by-trial co-variation between perceptual decisions and neuronal firing (choice probability (CP)). Macaque monkeys were trained to discriminate the direction of rotation of a cylinder, using the binocular depth between the moving dots that form its front and rear surfaces. We manipulated the activity of single neurons trial-to-trial by introducing task-irrelevant stimulus changes: dot motion in cylinders was aligned with neuronal preference on only half the trials, so that neurons were strongly activated with high firing rates on some trials and considerably less activated on others. We show that single neurons maintain high neurometric sensitivity for binocular depth in the face of substantial changes in firing rate. CP was correlated with neurometric sensitivity, not level of activation. In contrast, for individual neurons, the correlation between perceptual choice and neuronal activity may be fundamentally different when responding to different stimulus versions. Therefore, neuronal pools supporting sensory discrimination must be structured flexibly and independently for each stimulus configuration to be discriminated.This article is part of the themed issue ‘Vision in our three-dimensional world''.     

Changes in the acoustically evoked activity in the inferior colliculus of the rat after functional ablation of the auditory cortex

The role of the cortico-tectal pathways in the processing of auditory signals was investigated by recording the click-evoked responses and extracellular multiple unit activity in the inferior colliculus (IC) after functional ablation of the auditory cortex (AC) by local intracortical application of a sodium channel blocker, tetrodotoxin (TTX). Click-evoked IC responses (IC-ER) and multiple unit activity in response to tone bursts were recorded with implanted electrodes in the IC of rats lightly anaesthetized with xylazine. Neural activity was recorded before and after the application of TTX into the ipsilateral auditory cortex (AC) through three implanted cannulas in a total dose of 30 ng. The functional status of the AC was monitored by recording click-evoked middle latency responses from a ball electrode implanted on the AC. During inactivation of the AC, IC-ER amplitudes were either increased (48 % of the cases), decreased (32 % of the cases) or not evidently changed (20 % of the cases). Corresponding effects were observed in the firing rate of IC neurons. Functional ablation of the AC also resulted in a significant prolongation of the latencies of individual waves of the IC-ER. However, the discharge pattern of the multiple unit responses, response thresholds and tuning were not altered during AC inactivation. IC neural activity recovered within several hours, and maximally during 2 days. The results reveal principles of the interaction of cortico-tectal pathways with IC neuronal activity.     

Different processing phases for features, figures, and selective attention in the primary visual cortex

Our visual system imposes structure onto images that usually contain a diversity of surfaces, contours, and colors. Psychological theories propose that there are multiple steps in this process that occur in hierarchically organized regions of the cortex: early visual areas register basic features, higher areas bind them into objects, and yet higher areas select the objects that are relevant for behavior. Here we test these theories by recording from the primary visual cortex (area V1) of monkeys. We demonstrate that the V1 neurons first register the features (at a latency of 48 ms), then segregate figures from the background (after 57 ms), and finally select relevant figures over irrelevant ones (after 137 ms). We conclude that the psychological processing stages map onto distinct time episodes that unfold in the visual cortex after the presentation of a new stimulus, so that area V1 may contribute to all these processing steps.     

Coding of communication calls in the subcortical and cortical structures of the auditory system

The processing of species-specific communication signals in the auditory system represents an important aspect of animal behavior and is crucial for its social interactions, reproduction, and survival. In this article the neuronal mechanisms underlying the processing of communication signals in the higher centers of the auditory system--inferior colliculus (IC), medial geniculate body (MGB) and auditory cortex (AC)--are reviewed, with particular attention to the guinea pig. The selectivity of neuronal responses for individual calls in these auditory centers in the guinea pig is usually low--most neurons respond to calls as well as to artificial sounds; the coding of complex sounds in the central auditory nuclei is apparently based on the representation of temporal and spectral features of acoustical stimuli in neural networks. Neuronal response patterns in the IC reliably match the sound envelope for calls characterized by one or more short impulses, but do not exactly fit the envelope for long calls. Also, the main spectral peaks are represented by neuronal firing rates in the IC. In comparison to the IC, response patterns in the MGB and AC demonstrate a less precise representation of the sound envelope, especially in the case of longer calls. The spectral representation is worse in the case of low-frequency calls, but not in the case of broad-band calls. The emotional content of the call may influence neuronal responses in the auditory pathway, which can be demonstrated by stimulation with time-reversed calls or by measurements performed under different levels of anesthesia. The investigation of the principles of the neural coding of species-specific vocalizations offers some keys for understanding the neural mechanisms underlying human speech perception.     

Distributed representation of perceptual categories in the auditory cortex

Categorical perception is a process by which a continuous stimulus space is partitioned to represent discrete sensory events. Early experience has been shown to shape categorical perception and enlarge cortical representations of experienced stimuli in the sensory cortex. The present study examines the hypothesis that enlargement in cortical stimulus representations is a mechanism of categorical perception. Perceptual discrimination and identification behaviors were analyzed in model auditory cortices that incorporated sound exposure-induced plasticity effects. The model auditory cortex with over-representations of specific stimuli exhibited categorical perception behaviors for those specific stimuli. These results indicate that enlarged stimulus representations in the sensory cortex may be a mechanism for categorical perceptual learning.     

Increased Firing Irregularity as an Emergent Property of Neural-State Transition in Monkey Prefrontal Cortex

Flexible behaviors are organized by complex neural networks in the prefrontal cortex. Recent studies have suggested that such networks exhibit multiple dynamical states, and can switch rapidly from one state to another. In many complex systems such as the brain, the early-warning signals that may predict whether a critical threshold for state transitions is approaching are extremely difficult to detect. We hypothesized that increases in firing irregularity are a crucial measure for predicting state transitions in the underlying neuronal circuits of the prefrontal cortex. We used both experimental and theoretical approaches to test this hypothesis. Experimentally, we analyzed activities of neurons in the prefrontal cortex while monkeys performed a maze task that required them to perform actions to reach a goal. We observed increased firing irregularity before the activity changed to encode goal-to-action information. Theoretically, we constructed theoretical generic neural networks and demonstrated that changes in neuronal gain on functional connectivity resulted in a loss of stability and an altered state of the networks, accompanied by increased firing irregularity. These results suggest that assessing the temporal pattern of neuronal fluctuations provides important clues regarding the state stability of the prefrontal network. We also introduce a novel scheme that the prefrontal cortex functions in a metastable state near the critical point of bifurcation. According to this scheme, firing irregularity in the prefrontal cortex indicates that the system is about to change its state and the flow of information in a flexible manner, which is essential for executive functions. This metastable and/or critical dynamical state of the prefrontal cortex may account for distractibility and loss of flexibility in the prefrontal cortex in major mental illnesses such as schizophrenia.     

Diversity of Intrinsic Frequency Encoding Patterns in Rat Cortical Neurons—Mechanisms and Possible Functions

Extracellular recordings of single neurons in primary and secondary somatosensory cortices of monkeys in vivo have shown that their firing rate can increase, decrease, or remain constant in different cells, as the external stimulus frequency increases. We observed similar intrinsic firing patterns (increasing, decreasing or constant) in rat somatosensory cortex in vitro, when stimulated with oscillatory input using conductance injection (dynamic clamp). The underlying mechanism of this observation is not obvious, and presents a challenge for mathematical modelling. We propose a simple principle for describing this phenomenon using a leaky integrate-and-fire model with sinusoidal input, an intrinsic oscillation and Poisson noise. Additional enhancement of the gain of encoding could be achieved by local network connections amongst diverse intrinsic response patterns. Our work sheds light on the possible cellular and network mechanisms underlying these opposing neuronal responses, which serve to enhance signal detection.     

Maximum decoding abilities of temporal patterns and synchronized firings: application to auditory neurons responding to click trains and amplitude modulated white noise

Simultaneous recordings of an increasing number of neurons have recently become available, but few methods have been proposed to handle this activity. Here, we extract and investigate all the possible temporal neural activity patterns based on synchronized firings of neurons recorded on multiple electrodes, or based on bursts of single-electrode activity in cat primary auditory cortex. We apply this to responses to periodic click trains or sinusoïdal amplitude modulated noise by obtaining for each pattern its temporal modulation transfer function. An algorithm that maximizes the mutual information between all patterns and stimuli subsequently leads to the identification of patterns that optimally decode modulation frequency (MF). We show that stimulus information contained in multi-electrode synchronized firing is not redundant with single-electrode firings and leads to improved efficiency of MF decoding. We also show that the combined use of firing rate and temporal codes leads to a better discrimination of the MF.     

Neuromechanistic Model of Auditory Bistability

Sequences of higher frequency A and lower frequency B tones repeating in an ABA- triplet pattern are widely used to study auditory streaming. One may experience either an integrated percept, a single ABA-ABA- stream, or a segregated percept, separate but simultaneous streams A-A-A-A- and -B---B--. During minutes-long presentations, subjects may report irregular alternations between these interpretations. We combine neuromechanistic modeling and psychoacoustic experiments to study these persistent alternations and to characterize the effects of manipulating stimulus parameters. Unlike many phenomenological models with abstract, percept-specific competition and fixed inputs, our network model comprises neuronal units with sensory feature dependent inputs that mimic the pulsatile-like A1 responses to tones in the ABA- triplets. It embodies a neuronal computation for percept competition thought to occur beyond primary auditory cortex (A1). Mutual inhibition, adaptation and noise are implemented. We include slow NDMA recurrent excitation for local temporal memory that enables linkage across sound gaps from one triplet to the next. Percepts in our model are identified in the firing patterns of the neuronal units. We predict with the model that manipulations of the frequency difference between tones A and B should affect the dominance durations of the stronger percept, the one dominant a larger fraction of time, more than those of the weaker percept—a property that has been previously established and generalized across several visual bistable paradigms. We confirm the qualitative prediction with our psychoacoustic experiments and use the behavioral data to further constrain and improve the model, achieving quantitative agreement between experimental and modeling results. Our work and model provide a platform that can be extended to consider other stimulus conditions, including the effects of context and volition.     

Inferotemporal cortex subserves three-dimensional structure categorization

We perceive real-world objects as three-dimensional (3D), yet it is unknown which brain area underlies our ability to perceive objects in this way. The macaque inferotemporal (IT) cortex contains neurons that respond selectively to 3D structures defined by binocular disparity. To examine the causal role of IT in the categorization of 3D structures, we electrically stimulated clusters of IT neurons with a similar 3D-structure preference while monkeys performed a 3D-structure categorization task. Microstimulation of 3D-structure-selective IT clusters caused monkeys to choose the preferred structure of the 3D-structure-selective neurons considerably more often. Microstimulation in IT also accelerated the monkeys' choice for the preferred structure, while delaying choices corresponding to the nonpreferred structure of a given site. These findings reveal that 3D-structure-selective neurons in IT contribute to the categorization of 3D objects.     

Sparse representation of sounds in the unanesthetized auditory cortex

How do neuronal populations in the auditory cortex represent acoustic stimuli? Although sound-evoked neural responses in the anesthetized auditory cortex are mainly transient, recent experiments in the unanesthetized preparation have emphasized subpopulations with other response properties. To quantify the relative contributions of these different subpopulations in the awake preparation, we have estimated the representation of sounds across the neuronal population using a representative ensemble of stimuli. We used cell-attached recording with a glass electrode, a method for which single-unit isolation does not depend on neuronal activity, to quantify the fraction of neurons engaged by acoustic stimuli (tones, frequency modulated sweeps, white-noise bursts, and natural stimuli) in the primary auditory cortex of awake head-fixed rats. We find that the population response is sparse, with stimuli typically eliciting high firing rates (>20 spikes/second) in less than 5% of neurons at any instant. Some neurons had very low spontaneous firing rates (<0.01 spikes/second). At the other extreme, some neurons had driven rates in excess of 50 spikes/second. Interestingly, the overall population response was well described by a lognormal distribution, rather than the exponential distribution that is often reported. Our results represent, to our knowledge, the first quantitative evidence for sparse representations of sounds in the unanesthetized auditory cortex. Our results are compatible with a model in which most neurons are silent much of the time, and in which representations are composed of small dynamic subsets of highly active neurons.     

Some properties of auditory neuron’s model trained by firing caused by tones modulated by low-frequency noise

In many sensory systems the formation of burst firing can be observed along a way from the periphery to the central nuclei. We investigate the putative transformation of spontaneous activity in the auditory pathway using a neuron model trained by real firing recorded in the auditory nuclei of the frog. The model has 200 separate inputs (neuronal spines). It is supposed that every spine is a coincidence detector. Its output (synaptic potential) sharply increases at emergence of the precisely certain interpulse interval in an input pulse sequence. If the total synaptic potentials excess a threshold, the model generates output spike, which changes weight of all spines according to the simplified Hebb principle. The model was trained by real firing caused in the auditory nuclei of the frog by tones modulated by low-frequency noise in the frequency ranges of 0–15 Hz, 0–50 Hz or 0–150 Hz. After that training the synaptic weights of every spine essentially changed. Thus, along with some increase of weights of spines tuned to boundary frequencies of modulating noise, the most characteristic change was the emphasizing weights of spines tuned to short interpulse intervals. As a result the spontaneous activity passed through the trained model became much more bursting. Efficiency of a signal transmission in model was higher when input spontaneous activity of real cells contains bursts of spikes. Results of modeling are discussed in connection with modern physiological data demonstrating the functional advantage of bursting.     

Coding of Visual Object Features and Feature Conjunctions in the Human Brain

Object recognition is achieved through neural mechanisms reliant on the activity of distributed coordinated neural assemblies. In the initial steps of this process, an object''s features are thought to be coded very rapidly in distinct neural assemblies. These features play different functional roles in the recognition process - while colour facilitates recognition, additional contours and edges delay it. Here, we selectively varied the amount and role of object features in an entry-level categorization paradigm and related them to the electrical activity of the human brain. We found that early synchronizations (approx. 100 ms) increased quantitatively when more image features had to be coded, without reflecting their qualitative contribution to the recognition process. Later activity (approx. 200–400 ms) was modulated by the representational role of object features. These findings demonstrate that although early synchronizations may be sufficient for relatively crude discrimination of objects in visual scenes, they cannot support entry-level categorization. This was subserved by later processes of object model selection, which utilized the representational value of object features such as colour or edges to select the appropriate model and achieve identification.