Transforming bottom-up topographic representations with top-down signals in the brain

There has been considerable success in allocating function to the different parts of the brain. We also know much about brain organisation in different regions of the brain and how different brain regions connect to one another. One of the most important next steps for modern neuroscience is to work out how different areas of the brain interact with one another. In particular we need to know how sensory regions communicate with association areas and vice versa. This article explores how top-down signals originating from association areas may be used to process and transform bottom-up representations originating from sensory areas of the brain. Simple models of networks containing topographically organised ensembles of neurons used to integrate and process information are described. The different models can be used to process information in a variety of different ways that could be used as the starting point for a variety of cognitive operations, in particular the extraction of abstract information from sensory representations.     

Creating abstract topographic representations: Implications for coding,learning and reasoning

Topographic maps are a fundamental and ubiquitous feature of the sensory and motor regions of the brain. There is less evidence for the existence of conventional topographic maps in associational areas of the brain such as the prefrontal cortex and parietal cortex. The existence of topographically arranged anatomical projections is far more widespread and occurs in associational regions of the brain as well as sensory and motor regions: this points to a more widespread existence of topographically organised maps within associational cortex than currently recognised. Indeed, there is increasing evidence that abstract topographic representations may also occur in these regions. For example, a topographic mnemonic map of visual space has been described in the dorsolateral prefrontal cortex and topographically arranged visuospatial attentional signals have been described in parietal association cortex. This article explores how abstract representations might be extracted from sensory topographic representations and subsequently code abstract information. Finally a simple model is presented that shows how abstract topographic representations could be integrated with other information within the brain to solve problems or form abstract associations. The model uses correlative firing to detect associations between different types of stimuli. It is flexible because it can produce correlations between information represented in a topographic or non-topographic coordinate system. It is proposed that a similar process could be used in high-level cognitive operations such as learning and reasoning.     

Interactions between areas of the cortical grasping network

Skilled grasp is a sensorimotor process requiring the brain to extract sensory cues from the environment to shape a motor command. Although a large body of literature has focused on which brain areas either integrate the visual object's properties or control the motor output, it is still unclear how grasp-related information is transferred from one area to another. Understanding interactions between brain areas is crucial for the study of visuomotor transformations. Recently, new advances in both human and non-human primates have shown it is possible to study cortico-cortical interactions during different task contexts. This sheds new light on how brain areas are integrated in a dynamic network for controlling grasping actions.     

Dynamic construction of stimulus values in the ventromedial prefrontal cortex

Signals representing the value assigned to stimuli at the time of choice have been repeatedly observed in ventromedial prefrontal cortex (vmPFC). Yet it remains unknown how these value representations are computed from sensory and memory representations in more posterior brain regions. We used electroencephalography (EEG) while subjects evaluated appetitive and aversive food items to study how event-related responses modulated by stimulus value evolve over time. We found that value-related activity shifted from posterior to anterior, and from parietal to central to frontal sensors, across three major time windows after stimulus onset: 150-250 ms, 400-550 ms, and 700-800 ms. Exploratory localization of the EEG signal revealed a shifting network of activity moving from sensory and memory structures to areas associated with value coding, with stimulus value activity localized to vmPFC only from 400 ms onwards. Consistent with these results, functional connectivity analyses also showed a causal flow of information from temporal cortex to vmPFC. Thus, although value signals are present as early as 150 ms after stimulus onset, the value signals in vmPFC appear relatively late in the choice process, and seem to reflect the integration of incoming information from sensory and memory related regions.     

End-to-end neural system identification with neural information flow

Neural information flow (NIF) provides a novel approach for system identification in neuroscience. It models the neural computations in multiple brain regions and can be trained end-to-end via stochastic gradient descent from noninvasive data. NIF models represent neural information processing via a network of coupled tensors, each encoding the representation of the sensory input contained in a brain region. The elements of these tensors can be interpreted as cortical columns whose activity encodes the presence of a specific feature in a spatiotemporal location. Each tensor is coupled to the measured data specific to a brain region via low-rank observation models that can be decomposed into the spatial, temporal and feature receptive fields of a localized neuronal population. Both these observation models and the convolutional weights defining the information processing within regions are learned end-to-end by predicting the neural signal during sensory stimulation. We trained a NIF model on the activity of early visual areas using a large-scale fMRI dataset recorded in a single participant. We show that we can recover plausible visual representations and population receptive fields that are consistent with empirical findings.     

Self-localization and the entorhinal-hippocampal system

Self-localization requires that information from several sensory modalities and knowledge domains be integrated in order to identify an environment and determine current location and heading. This integration occurs by the convergence of highly processed sensory information onto neural systems in entorhinal cortex and hippocampus. Entorhinal neurons combine angular and linear self-motion information to generate an oriented metric signal that is then 'attached' to each environment using information about landmarks and context. Neurons in hippocampus use this signal to determine the animal's unique position within a particular environment. Elucidating this process illuminates not only spatial processing but also, more generally, how the brain builds knowledge representations from inputs carrying heterogeneous sensory and semantic content.     

State-Dependent Changes in Auditory Sensory Gating in Different Cortical Areas in Rats

Sensory gating is a process in which the brain’s response to a repetitive stimulus is attenuated; it is thought to contribute to information processing by enabling organisms to filter extraneous sensory inputs from the environment. To date, sensory gating has typically been used to determine whether brain function is impaired, such as in individuals with schizophrenia or addiction. In healthy subjects, sensory gating is sensitive to a subject’s behavioral state, such as acute stress and attention. The cortical response to sensory stimulation significantly decreases during sleep; however, information processing continues throughout sleep, and an auditory evoked potential (AEP) can be elicited by sound. It is not known whether sensory gating changes during sleep. Sleep is a non-uniform process in the whole brain with regional differences in neural activities. Thus, another question arises concerning whether sensory gating changes are uniform in different brain areas from waking to sleep. To address these questions, we used the sound stimuli of a Conditioning-testing paradigm to examine sensory gating during waking, rapid eye movement (REM) sleep and Non-REM (NREM) sleep in different cortical areas in rats. We demonstrated the following: 1. Auditory sensory gating was affected by vigilant states in the frontal and parietal areas but not in the occipital areas. 2. Auditory sensory gating decreased in NREM sleep but not REM sleep from waking in the frontal and parietal areas. 3. The decreased sensory gating in the frontal and parietal areas during NREM sleep was the result of a significant increase in the test sound amplitude.     

Multisensory Causal Inference in the Brain

At any given moment, our brain processes multiple inputs from its different sensory modalities (vision, hearing, touch, etc.). In deciphering this array of sensory information, the brain has to solve two problems: (1) which of the inputs originate from the same object and should be integrated and (2) for the sensations originating from the same object, how best to integrate them. Recent behavioural studies suggest that the human brain solves these problems using optimal probabilistic inference, known as Bayesian causal inference. However, how and where the underlying computations are carried out in the brain have remained unknown. By combining neuroimaging-based decoding techniques and computational modelling of behavioural data, a new study now sheds light on how multisensory causal inference maps onto specific brain areas. The results suggest that the complexity of neural computations increases along the visual hierarchy and link specific components of the causal inference process with specific visual and parietal regions.     

Phase-Coherence Transitions and Communication in the Gamma Range between Delay-Coupled Neuronal Populations

Synchronization between neuronal populations plays an important role in information transmission between brain areas. In particular, collective oscillations emerging from the synchronized activity of thousands of neurons can increase the functional connectivity between neural assemblies by coherently coordinating their phases. This synchrony of neuronal activity can take place within a cortical patch or between different cortical regions. While short-range interactions between neurons involve just a few milliseconds, communication through long-range projections between different regions could take up to tens of milliseconds. How these heterogeneous transmission delays affect communication between neuronal populations is not well known. To address this question, we have studied the dynamics of two bidirectionally delayed-coupled neuronal populations using conductance-based spiking models, examining how different synaptic delays give rise to in-phase/anti-phase transitions at particular frequencies within the gamma range, and how this behavior is related to the phase coherence between the two populations at different frequencies. We have used spectral analysis and information theory to quantify the information exchanged between the two networks. For different transmission delays between the two coupled populations, we analyze how the local field potential and multi-unit activity calculated from one population convey information in response to a set of external inputs applied to the other population. The results confirm that zero-lag synchronization maximizes information transmission, although out-of-phase synchronization allows for efficient communication provided the coupling delay, the phase lag between the populations, and the frequency of the oscillations are properly matched.     

Attention control in the primate brain

Attention is fundamental to all cognition. In the primate brain, it is implemented by a large-scale network that consists of areas spanning across all major lobes, also including subcortical regions. Classical attention accounts assume that control over the selection process in this network is exerted by ‘top-down’ mechanisms in the fronto-parietal cortex that influence sensory representations via feedback signals. More recent studies have expanded this view of attentional control. In this review, we will start from a traditional top-down account of attention control, and then discuss more recent findings on feature-based attention, thalamic influences, temporal network dynamics, and behavioral dynamics that collectively lead to substantial modifications. We outline how the different emerging accounts can be reconciled and integrated into a unified theory.     

Neural correlates of decisions

Once considered the province of philosophy and the behavioral sciences, the process of making decisions has received increasing scrutiny from neurobiologists. Recent research suggests that sensory judgements unfold through the gradual accumulation of neuronal signals in sensory-motor pathways, favoring one alternative over others. Stored representations of the outcome of prior actions activate neurons in many of these same areas during decision-making. The challenge for neurobiologists lies in deciphering how signals from these disparate areas are integrated to form a single behavioral choice and the mechanisms responsible for selecting the appropriate information upon which decisions should be informed in particular contexts.     

Audio-Tactile Integration and the Influence of Musical Training

Perception of our environment is a multisensory experience; information from different sensory systems like the auditory, visual and tactile is constantly integrated. Complex tasks that require high temporal and spatial precision of multisensory integration put strong demands on the underlying networks but it is largely unknown how task experience shapes multisensory processing. Long-term musical training is an excellent model for brain plasticity because it shapes the human brain at functional and structural levels, affecting a network of brain areas. In the present study we used magnetoencephalography (MEG) to investigate how audio-tactile perception is integrated in the human brain and if musicians show enhancement of the corresponding activation compared to non-musicians. Using a paradigm that allowed the investigation of combined and separate auditory and tactile processing, we found a multisensory incongruency response, generated in frontal, cingulate and cerebellar regions, an auditory mismatch response generated mainly in the auditory cortex and a tactile mismatch response generated in frontal and cerebellar regions. The influence of musical training was seen in the audio-tactile as well as in the auditory condition, indicating enhanced higher-order processing in musicians, while the sources of the tactile MMN were not influenced by long-term musical training. Consistent with the predictive coding model, more basic, bottom-up sensory processing was relatively stable and less affected by expertise, whereas areas for top-down models of multisensory expectancies were modulated by training.     

Sensory Perception: Lessons from Synesthesia: Using Synesthesia to Inform the Understanding of Sensory Perception

Synesthesia, the conscious, idiosyncratic, repeatable, and involuntary sensation of one sensory modality in response to another, is a condition that has puzzled both researchers and philosophers for centuries. Much time has been spent proving the condition’s existence as well as investigating its etiology, but what can be learned from synesthesia remains a poorly discussed topic. Here, synaesthesia is presented as a possible answer rather than a question to the current gaps in our understanding of sensory perception. By first appreciating the similarities between normal sensory perception and synesthesia, one can use what is known about synaesthesia, from behavioral and imaging studies, to inform our understanding of “normal” sensory perception. In particular, in considering synesthesia, one can better understand how and where the different sensory modalities interact in the brain, how different sensory modalities can interact without confusion ― the binding problem ― as well as how sensory perception develops.     

Symbols as Self-emergent Entities in an Optimization Process of Feature Extraction and Predictions

In the mammalian cortex the early sensory processing can be characterized as feature extraction resulting in local and analogue low-level representations. As a direct consequence, these map directly to the environment, but interpretation under natural conditions is ambiguous. In contrast, high-level representations for cognitive processing, e.g. language, require symbolic representations characterized by expression and syntax. The representations are binary, structured and disambiguated. However, do these fundamental functional distinctions translate into a fundamental distinction of the respective brain areas and their anatomical and physiological properties? Here we argue that the distinction between early sensory processing and higher cognitive functions may not be based on structural differences of cortical areas; instead similar learning principles acting on input signals with different statistics give rise to the observed variations of function. Firstly, we give an account of present research describing neuronal properties at early stages of sensory systems as a consequence of an optimization process over the set of natural stimuli. Secondly, addressing a stage following early visual processing we suggest to extend the unsupervised learning scheme by including predictive processes. These contain the widely used objective of temporal coherence as a special case and are a powerful approach to resolve ambiguities. Furthermore, in combination with a prior on the bandwidth of information exchange between units it leads to a condensation of information. Thirdly, as a crucial step, not only are predictive units optimized, but the selectivity of the feature extractors are adapted to allow optimal predictability. Thus, over and beyond making useful predictions, we propose that the predictability of a stimulus be in itself a selection criterion for further processing. In a hierarchical system the combined optimization process leads to entities that represent condensed pieces of knowledge and that are not analogue anymore. Instead, these entities work as arguments in a framework of transformations that realize predictions. Thus, the criteria of predictability and condensation in an optimization of sensory representations relate directly to the two defining properties of symbols of expression and syntax. In this paper, we sketch an unsupervised learning process that gradually transforms analogue local representations into discrete binary representations by means of four hypotheses. We propose that in this optimization process acting in a hierarchical system, entities emerge at, higher levels that fulfil the criteria defining symbols, instantiating qualitatively different representations at similarly structured low and high levels.     

Changes in cerebral functional organization during cognitive development

It has been just under a decade since contemporary neuroimaging tools, such as functional magnetic resonance imaging, were first applied to developmental questions. These tools provide invaluable information on how brain anatomy, function and connectivity change during development. Studies using these methods with children and adolescents show that brain regions that support motor and sensory function mature earliest, whereas higher-order association areas, such as the prefrontal cortex, which integrate these functions, mature later.     

Short-term depression and transient memory in sensory cortex

Persistent neuronal activity is usually studied in the context of short-term memory localized in central cortical areas. Recent studies show that early sensory areas also can have persistent representations of stimuli which emerge quickly (over tens of milliseconds) and decay slowly (over seconds). Traditional positive feedback models cannot explain sensory persistence for at least two reasons: (i) They show attractor dynamics, with transient perturbations resulting in a quasi-permanent change of system state, whereas sensory systems return to the original state after a transient. (ii) As we show, those positive feedback models which decay to baseline lose their persistence when their recurrent connections are subject to short-term depression, a common property of excitatory connections in early sensory areas. Dual time constant network behavior has also been implemented by nonlinear afferents producing a large transient input followed by much smaller steady state input. We show that such networks require unphysiologically large onset transients to produce the rise and decay observed in sensory areas. Our study explores how memory and persistence can be implemented in another model class, derivative feedback networks. We show that these networks can operate with two vastly different time courses, changing their state quickly when new information is coming in but retaining it for a long time, and that these capabilities are robust to short-term depression. Specifically, derivative feedback networks with short-term depression that acts differentially on positive and negative feedback projections are capable of dynamically changing their time constant, thus allowing fast onset and slow decay of responses without requiring unrealistically large input transients.     

Multimodal spatial representations engaged in human parietal cortex during both saccadic and manual spatial orienting

BACKGROUND: Recent neuroimaging studies have found that several areas of the human brain, including parietal regions, can respond multimodally. But given single-cell evidence that responses in primate parietal cortex can be motor-related, some of the human multimodal activations might reflect convergent activation of potentially motor-related areas, rather than multimodal representations of space independent of motor factors. Here we crossed sensory stimulation of different modalities (vision or touch, in left or right hemifield) with spatially directed responses to such stimulation by different effector-systems (saccadic or manual). RESULTS: The fMRI results revealed representations of contralateral space in both the posterior part of the superior parietal gyrus and the anterior intraparietal sulcus that activated independently of both sensory modality and motor response. Multimodal saccade-related or manual-related activations were found, by contrast, in different regions of parietal cortex. CONCLUSIONS: Whereas some parietal regions have specific motor functions, others are engaged during the execution of movements to the contralateral hemifield irrespective of both input modality and the type of motor effector.     

Multisensory interplay reveals crossmodal influences on 'sensory-specific' brain regions, neural responses, and judgments

Although much traditional sensory research has studied each sensory modality in isolation, there has been a recent explosion of interest in causal interplay between different senses. Various techniques have now identified numerous multisensory convergence zones in the brain. Some convergence may arise surprisingly close to low-level sensory-specific cortex, and some direct connections may exist even between primary sensory cortices. A variety of multisensory phenomena have now been reported in which sensory-specific brain responses and perceptual judgments concerning one sense can be affected by relations with other senses. We survey recent progress in this multisensory field, foregrounding human studies against the background of invasive animal work and highlighting possible underlying mechanisms. These include rapid feedforward integration, possible thalamic influences, and/or feedback from multisensory regions to sensory-specific brain areas. Multisensory interplay is more prevalent than classic modular approaches assumed, and new methods are now available to determine the underlying circuits.     

Temporal codes and sparse representations: A key to understanding rapid processing in the visual system

Where neural information processing is concerned, there is no debate about the fact that spikes are the basic currency for transmitting information between neurons. How the brain actually uses them to encode information remains more controversial. It is commonly assumed that neuronal firing rate is the key variable, but the speed with which images can be analysed by the visual system poses a major challenge for rate-based approaches. We will thus expose here the possibility that the brain makes use of the spatio-temporal structure of spike patterns to encode information. We then consider how such rapid selective neural responses can be generated rapidly through spike-timing-dependent plasticity (STDP) and how these selectivities can be used for visual representation and recognition. Finally, we show how temporal codes and sparse representations may very well arise one from another and explain some of the remarkable features of processing in the visual system.     

Multisensory Oddity Detection as Bayesian Inference

A key goal for the perceptual system is to optimally combine information from all the senses that may be available in order to develop the most accurate and unified picture possible of the outside world. The contemporary theoretical framework of ideal observer maximum likelihood integration (MLI) has been highly successful in modelling how the human brain combines information from a variety of different sensory modalities. However, in various recent experiments involving multisensory stimuli of uncertain correspondence, MLI breaks down as a successful model of sensory combination. Within the paradigm of direct stimulus estimation, perceptual models which use Bayesian inference to resolve correspondence have recently been shown to generalize successfully to these cases where MLI fails. This approach has been known variously as model inference, causal inference or structure inference. In this paper, we examine causal uncertainty in another important class of multi-sensory perception paradigm – that of oddity detection and demonstrate how a Bayesian ideal observer also treats oddity detection as a structure inference problem. We validate this approach by showing that it provides an intuitive and quantitative explanation of an important pair of multi-sensory oddity detection experiments – involving cues across and within modalities – for which MLI previously failed dramatically, allowing a novel unifying treatment of within and cross modal multisensory perception. Our successful application of structure inference models to the new ‘oddity detection’ paradigm, and the resultant unified explanation of across and within modality cases provide further evidence to suggest that structure inference may be a commonly evolved principle for combining perceptual information in the brain.