Distributed representation of perceptual categories in the auditory cortex

Categorical perception is a process by which a continuous stimulus space is partitioned to represent discrete sensory events. Early experience has been shown to shape categorical perception and enlarge cortical representations of experienced stimuli in the sensory cortex. The present study examines the hypothesis that enlargement in cortical stimulus representations is a mechanism of categorical perception. Perceptual discrimination and identification behaviors were analyzed in model auditory cortices that incorporated sound exposure-induced plasticity effects. The model auditory cortex with over-representations of specific stimuli exhibited categorical perception behaviors for those specific stimuli. These results indicate that enlarged stimulus representations in the sensory cortex may be a mechanism for categorical perceptual learning.     

Learning in mammalian sensory cortex

The improvement of perceptual capabilities with training can offer important insight into the physiological basis of learning in the cerebral cortex. The rapid time course and ease with which some perceptual capabilities can improve suggest that learning is an integral part of normal perception. Electrophysiological and neuroimaging studies of sensory systems in the cortex suggest that the changes underlying perceptual learning can occur in a variety of areas and are likely to involve multiple mechanisms. In particular, recent psychological and physiological studies suggest that perceptual learning might often involve the task-specific suppression of signals that interfere with performance.     

Transient attention enhances perceptual performance and FMRI response in human visual cortex

When a visual stimulus suddenly appears, it captures attention, producing a transient improvement of performance on basic visual tasks. We investigate the effect of transient attention on stimulus representations in early visual areas using rapid event-related fMRI. Participants discriminated the orientation of one of two gratings preceded or followed by a nonpredictive peripheral cue. Compared to control conditions, precueing the target location improved performance and produced a larger fMRI response in corresponding retinotopic areas. This enhancement progressively increased from striate to extrastriate areas. Control conditions indicated that the enhanced fMRI response was not due to sensory summation of cue and target signals. Thus, an uninformative precue increases both perceptual performance and the concomitant stimulus-evoked activity in early visual areas. These results provide evidence regarding the retinotopically specific neural correlate for the effects of transient attention on early vision.     

Symbols as Self-emergent Entities in an Optimization Process of Feature Extraction and Predictions

In the mammalian cortex the early sensory processing can be characterized as feature extraction resulting in local and analogue low-level representations. As a direct consequence, these map directly to the environment, but interpretation under natural conditions is ambiguous. In contrast, high-level representations for cognitive processing, e.g. language, require symbolic representations characterized by expression and syntax. The representations are binary, structured and disambiguated. However, do these fundamental functional distinctions translate into a fundamental distinction of the respective brain areas and their anatomical and physiological properties? Here we argue that the distinction between early sensory processing and higher cognitive functions may not be based on structural differences of cortical areas; instead similar learning principles acting on input signals with different statistics give rise to the observed variations of function. Firstly, we give an account of present research describing neuronal properties at early stages of sensory systems as a consequence of an optimization process over the set of natural stimuli. Secondly, addressing a stage following early visual processing we suggest to extend the unsupervised learning scheme by including predictive processes. These contain the widely used objective of temporal coherence as a special case and are a powerful approach to resolve ambiguities. Furthermore, in combination with a prior on the bandwidth of information exchange between units it leads to a condensation of information. Thirdly, as a crucial step, not only are predictive units optimized, but the selectivity of the feature extractors are adapted to allow optimal predictability. Thus, over and beyond making useful predictions, we propose that the predictability of a stimulus be in itself a selection criterion for further processing. In a hierarchical system the combined optimization process leads to entities that represent condensed pieces of knowledge and that are not analogue anymore. Instead, these entities work as arguments in a framework of transformations that realize predictions. Thus, the criteria of predictability and condensation in an optimization of sensory representations relate directly to the two defining properties of symbols of expression and syntax. In this paper, we sketch an unsupervised learning process that gradually transforms analogue local representations into discrete binary representations by means of four hypotheses. We propose that in this optimization process acting in a hierarchical system, entities emerge at, higher levels that fulfil the criteria defining symbols, instantiating qualitatively different representations at similarly structured low and high levels.     

The neural basis of perceptual learning

Perceptual learning is a lifelong process. We begin by encoding information about the basic structure of the natural world and continue to assimilate information about specific patterns with which we become familiar. The specificity of the learning suggests that all areas of the cerebral cortex are plastic and can represent various aspects of learned information. The neural substrate of perceptual learning relates to the nature of the neural code itself, including changes in cortical maps, in the temporal characteristics of neuronal responses, and in modulation of contextual influences. Top-down control of these representations suggests that learning involves an interaction between multiple cortical areas.     

A Hierarchy of Time-Scales and the Brain

In this paper, we suggest that cortical anatomy recapitulates the temporal hierarchy that is inherent in the dynamics of environmental states. Many aspects of brain function can be understood in terms of a hierarchy of temporal scales at which representations of the environment evolve. The lowest level of this hierarchy corresponds to fast fluctuations associated with sensory processing, whereas the highest levels encode slow contextual changes in the environment, under which faster representations unfold. First, we describe a mathematical model that exploits the temporal structure of fast sensory input to track the slower trajectories of their underlying causes. This model of sensory encoding or perceptual inference establishes a proof of concept that slowly changing neuronal states can encode the paths or trajectories of faster sensory states. We then review empirical evidence that suggests that a temporal hierarchy is recapitulated in the macroscopic organization of the cortex. This anatomic-temporal hierarchy provides a comprehensive framework for understanding cortical function: the specific time-scale that engages a cortical area can be inferred by its location along a rostro-caudal gradient, which reflects the anatomical distance from primary sensory areas. This is most evident in the prefrontal cortex, where complex functions can be explained as operations on representations of the environment that change slowly. The framework provides predictions about, and principled constraints on, cortical structure–function relationships, which can be tested by manipulating the time-scales of sensory input.     

Learning to smell the roses: experience-dependent neural plasticity in human piriform and orbitofrontal cortices

It is widely presumed that odor quality is a direct outcome of odorant structure, but human studies indicate that molecular knowledge of an odorant is not always sufficient to predict odor quality. Indeed, the same olfactory input may generate different odor percepts depending on prior learning and experience. Combining functional magnetic resonance imaging with an olfactory paradigm of perceptual learning, we examined how sensory experience modifies odor perception and odor quality coding in the human brain. Prolonged exposure to a target odorant enhanced perceptual differentiation for odorants related in odor quality or functional group, an effect that was paralleled by learning-induced response increases in piriform cortex and orbitofrontal cortex (OFC). Critically, the magnitude of OFC activation predicted subsequent improvement in behavioral differentiation. Our findings suggest that neural representations of odor quality can be rapidly updated through mere perceptual experience, a mechanism that may underlie the development of odor perception.     

Predicting the stream of consciousness from activity in human visual cortex

Can the rapid stream of conscious experience be predicted from brain activity alone? Recently, spatial patterns of activity in visual cortex have been successfully used to predict feature-specific stimulus representations for both visible and invisible stimuli. However, because these studies examined only the prediction of static and unchanging perceptual states during extended periods of stimulation, it remains unclear whether activity in early visual cortex can also predict the rapidly and spontaneously changing stream of consciousness. Here, we used binocular rivalry to induce frequent spontaneous and stochastic changes in conscious experience without any corresponding changes in sensory stimulation, while measuring brain activity with fMRI. Using information that was present in the multivariate pattern of responses to stimulus features, we could accurately predict, and therefore track, participants' conscious experience from the fMRI signal alone while it underwent many spontaneous changes. Prediction in primary visual cortex primarily reflected eye-based signals, whereas prediction in higher areas reflected the color of the percept. Furthermore, accurate prediction during binocular rivalry could be established with signals recorded during stable monocular viewing, showing that prediction generalized across viewing conditions and did not require or rely on motor responses. It is therefore possible to predict the dynamically changing time course of subjective experience with only brain activity.     

Reshaping sensory representations by task-specific brain states: Toward cortical circuit mechanisms

Perception is internally constructed by integrating brain states with external sensory inputs, a process depending on the topdown modulation of sensory representations. A wealth of earlier studies described task-dependent modulations of sensory cortex corroborating perceptual and behavioral phenomena. But only with recent technological advancements, the underlying circuit-level mechanisms began to be unveiled. We review recent progress along this line of research. It begins to be appreciated that topdown signals can encode various types of task-related information, ranging from task engagement, and category knowledge to decision execution; these signals are transferred via feedback pathways originating from distinct association cortices and interact with sensory cortical circuits. These plausible mechanisms support a broad range of computations from predictive coding to inference making, ultimately form dynamic percepts and endow behavioral flexibility.     

Learning to hear: plasticity of auditory cortical processing

Sensory experience and auditory cortex plasticity are intimately related. This relationship is most striking during infancy when changes in sensory input can have profound effects on the functional organization of the developing cortex. But a considerable degree of plasticity is retained throughout life, as demonstrated by the cortical reorganization that follows damage to the sensory periphery or by the more controversial changes in response properties that are thought to accompany perceptual learning. Recent studies in the auditory system have revealed the remarkably adaptive nature of sensory processing and provided important insights into the way in which cortical circuits are shaped by experience and learning.     

Reward Maximization Justifies the Transition from Sensory Selection at Childhood to Sensory Integration at Adulthood

In a multisensory task, human adults integrate information from different sensory modalities -behaviorally in an optimal Bayesian fashion- while children mostly rely on a single sensor modality for decision making. The reason behind this change of behavior over age and the process behind learning the required statistics for optimal integration are still unclear and have not been justified by the conventional Bayesian modeling. We propose an interactive multisensory learning framework without making any prior assumptions about the sensory models. In this framework, learning in every modality and in their joint space is done in parallel using a single-step reinforcement learning method. A simple statistical test on confidence intervals on the mean of reward distributions is used to select the most informative source of information among the individual modalities and the joint space. Analyses of the method and the simulation results on a multimodal localization task show that the learning system autonomously starts with sensory selection and gradually switches to sensory integration. This is because, relying more on modalities -i.e. selection- at early learning steps (childhood) is more rewarding than favoring decisions learned in the joint space since, smaller state-space in modalities results in faster learning in every individual modality. In contrast, after gaining sufficient experiences (adulthood), the quality of learning in the joint space matures while learning in modalities suffers from insufficient accuracy due to perceptual aliasing. It results in tighter confidence interval for the joint space and consequently causes a smooth shift from selection to integration. It suggests that sensory selection and integration are emergent behavior and both are outputs of a single reward maximization process; i.e. the transition is not a preprogrammed phenomenon.     

Perceptual learning and sensomotor flexibility: cortical plasticity under attentional control?

Recent research reveals long-lasting cortical plasticity of early sensory cortices even in adults. Sensory signals could be modified under top-down control if necessary quite early in order to optimize their signal-to-noise ratio, leading to 'low level' or 'early' perceptual learning (PL). For easy tasks, such elaborate top-down influences are usually not required, and learning is restricted to late selection of the appropriate signals on higher cortical levels, which seems easier and faster to achieve. But to reach the absolute limits of sensory performance, PL seems to optimize the entire chain of sensory processing. Hence, improvement for these extreme perceptual abilities is quite specific for a number of stimulus parameters, such as the position in the visual field and sometimes even the trained eye, reflecting the specificity of receptive fields in early sensory cortices. Early PL may be just one example--even if a very extensive one--of the mechanisms of neuronal plasticity and sensomotor flexibility that are constantly updating our sensomotor representations as a result of experience. As an illustration, this review contains some new experimental results on PL and sensory flexibility in the context of adaptation to multifocal intraocular lenses.     

Perceptual learning reduces interneuronal correlations in macaque visual cortex

Responses of neurons in early visual cortex change little with training and appear insufficient to account for perceptual learning. Behavioral performance, however, relies on population activity, and the accuracy of a population code is constrained by correlated noise among neurons. We tested whether training changes interneuronal correlations in the dorsal medial superior temporal area, which is involved in multisensory heading perception. Pairs of single units were recorded simultaneously in two groups of subjects: animals trained extensively in a heading discrimination task, and "naive" animals that performed a passive fixation task. Correlated noise was significantly weaker in trained versus naive animals, which might be expected to improve coding efficiency. However, we show that the observed uniform reduction in noise correlations leads to little change in population coding efficiency when all neurons are decoded. Thus, global changes in correlated noise among sensory neurons may be insufficient to account for perceptual learning.     

Learning through maps: functional significance of topographic organization in primary sensory cortex.

The presence of "maps" in sensory cortex is a hallmark of the mammalian nervous system, but the functional significance of topographic organization has been called into question by physiological studies claiming that patterns of neural behavioral activity transcend topographic boundaries. This paper discusses recent behavioral and physiological studies suggesting that, when animals or human subjects learn perceptual tasks, the neural modifications associated with the learning are distributed according to the spatial arrangement of the primary sensory cortical map. Topographical cortical representations of sensory events, therefore, appear to constitute a true structural framework for information processing and plasticity.     

Perceptual learning: specificity versus generalization

Perceptual learning improves performance on many tasks, from orientation discrimination to the identification of faces. Although conventional wisdom considered sensory cortices as hard-wired, the specificity of improvement achieved through perceptual learning indicates an involvement of early sensory cortices. These cortices might be more plastic than previously assumed, and both sum-potential and single cell recordings indeed demonstrate plasticity of neuronal responses of these sensory cortices. However, for learning to be optimally useful, it must generalize to other tasks. Further research on perceptual learning should therefore, in my opinion, investigate first, the conditions for generalization of training-induced improvement, second, its use for teaching and rehabilitation, and third, its dependence on pharmacological agents.     

Neural plasticity in high-level visual cortex underlying object perceptual learning

With intensive training, human can achieve impressive behavioral improvement on various perceptual tasks. This phenomenon, termed perceptual learning, has long been considered as a hallmark of the plasticity of sensory neural system. Not surprisingly, high-level vision, such as object perception, can also be improved by perceptual learning. Here we review recent psychophysical, electrophysiological, and neuroimaging studies investigating the effects of training on object selective cortex, such as monkey inferior temporal cortex and human lateral occipital area. Evidences show that learning leads to an increase in object selectivity at the single neuron level and/or the neuronal population level. These findings indicate that high-level visual cortex in humans is highly plastic and visual experience can strongly shape neural functions of these areas. At the end of the review, we discuss several important future directions in this area.     

Representational plasticity in the mammalian brain cortex. (Review article)

The epithelial receptors are represented in the mammalian brain cortex in a genetically defined, strictly regulated manner. Until the 1970s, the cortical maps and the wiring of the central nervous system were thought to be rather static and unchangeable. Subsequently, however, studies of sensory and motor cortical maps in particular genetic strains of animals and in animals with different perinatal or adult histories have revealed that the map organization can be modified at any time between conception and death. Especially studies of the effects of peripheral and central lesions and of perceptual learning on the sensory and motor cortical representations have had a dramatic effect in alerting neuroscientists and therapists to the reorganizational capacity of the adult brain. From a theoretical aspect, these changes in the cortical maps provide useful models for an understanding of the changes that can occur in the integrative functions of complex brain networks throughout life.     

Different dynamics of performance and brain activation in the time course of perceptual learning

Perceptual learning is regarded as a manifestation of experience-dependent plasticity in the sensory systems, yet the underlying neural mechanisms remain unclear. We measured the dynamics of performance on a visual task and brain activation in the human primary visual cortex (V1) across the time course of perceptual learning. Within the first few weeks of training, brain activation in a V1 subregion corresponding to the trained visual field quadrant and task performance both increased. However, while performance levels then saturated and were maintained at a constant level, brain activation in the corresponding areas decreased to the level observed before training. These findings indicate that there are distinct temporal phases in the time course of perceptual learning, related to differential dynamics of BOLD activity in visual cortex.     

Coding of cognitive magnitude: compressed scaling of numerical information in the primate prefrontal cortex

Whether cognitive representations are better conceived as language-based, symbolic representations or perceptually related, analog representations is a subject of debate. If cognitive processes parallel perceptual processes, then fundamental psychophysical laws should hold for each. To test this, we analyzed both behavioral and neuronal representations of numerosity in the prefrontal cortex of rhesus monkeys. The data were best described by a nonlinearly compressed scaling of numerical information, as postulated by the Weber-Fechner law or Stevens' law for psychophysical/sensory magnitudes. This nonlinear compression was observed on the neural level during the acquisition phase of the task and maintained through the memory phase with no further compression. These results suggest that certain cognitive and perceptual/sensory representations share the same fundamental mechanisms and neural coding schemes.     

A theory of cortical responses

This article concerns the nature of evoked brain responses and the principles underlying their generation. We start with the premise that the sensory brain has evolved to represent or infer the causes of changes in its sensory inputs. The problem of inference is well formulated in statistical terms. The statistical fundaments of inference may therefore afford important constraints on neuronal implementation. By formulating the original ideas of Helmholtz on perception, in terms of modern-day statistical theories, one arrives at a model of perceptual inference and learning that can explain a remarkable range of neurobiological facts.It turns out that the problems of inferring the causes of sensory input (perceptual inference) and learning the relationship between input and cause (perceptual learning) can be resolved using exactly the same principle. Specifically, both inference and learning rest on minimizing the brain's free energy, as defined in statistical physics. Furthermore, inference and learning can proceed in a biologically plausible fashion. Cortical responses can be seen as the brain's attempt to minimize the free energy induced by a stimulus and thereby encode the most likely cause of that stimulus. Similarly, learning emerges from changes in synaptic efficacy that minimize the free energy, averaged over all stimuli encountered. The underlying scheme rests on empirical Bayes and hierarchical models of how sensory input is caused. The use of hierarchical models enables the brain to construct prior expectations in a dynamic and context-sensitive fashion. This scheme provides a principled way to understand many aspects of cortical organization and responses. The aim of this article is to encompass many apparently unrelated anatomical, physiological and psychophysical attributes of the brain within a single theoretical perspective.In terms of cortical architectures, the theoretical treatment predicts that sensory cortex should be arranged hierarchically, that connections should be reciprocal and that forward and backward connections should show a functional asymmetry (forward connections are driving, whereas backward connections are both driving and modulatory). In terms of synaptic physiology, it predicts associative plasticity and, for dynamic models, spike-timing-dependent plasticity. In terms of electrophysiology, it accounts for classical and extra classical receptive field effects and long-latency or endogenous components of evoked cortical responses. It predicts the attenuation of responses encoding prediction error with perceptual learning and explains many phenomena such as repetition suppression, mismatch negativity (MMN) and the P300 in electroencephalography. In psychophysical terms, it accounts for the behavioural correlates of these physiological phenomena, for example, priming and global precedence. The final focus of this article is on perceptual learning as measured with the MMN and the implications for empirical studies of coupling among cortical areas using evoked sensory responses.