Dissecting neural circuits for multisensory integration and crossmodal processing

We rely on rich and complex sensory information to perceive and understand our environment. Our multisensory experience of the world depends on the brain''s remarkable ability to combine signals across sensory systems. Behavioural, neurophysiological and neuroimaging experiments have established principles of multisensory integration and candidate neural mechanisms. Here we review how targeted manipulation of neural activity using invasive and non-invasive neuromodulation techniques have advanced our understanding of multisensory processing. Neuromodulation studies have provided detailed characterizations of brain networks causally involved in multisensory integration. Despite substantial progress, important questions regarding multisensory networks remain unanswered. Critically, experimental approaches will need to be combined with theory in order to understand how distributed activity across multisensory networks collectively supports perception.     

Learning related interactions among neuronal systems involved in memory processes

Functional neuroimaging techniques using positron emission tomography (PET) and functional magnetic resonance imaging (fMRI) have provided new insights in our understanding of brain function from the molecular to the systems level. While subtraction strategy based data analyses have revealed the involvement of distributed brain regions in memory processes, covariance analysis based data analysis strategies allow functional interactions between brain regions of a neuronal network to be assessed. The focus of this chapter is to (1) establish the functional topography of episodic and working memory processes in young and old normal volunteers, (2) to assess functional interactions between modules of networks of brain regions by means of covariance based analyses and systems level modelling and (3) to relate neuroimaging data to the underpinning neural networks. Male normal young and old volunteers without neurological or psychiatric illness participated in neuroimaging studies (PET, fMRI) on working and episodic memory. Distributed brain areas are involved in memory processes (episodic and working memory) in young volunteers and show much of an overlap with respect to the network components. Systems level modelling analyses support the hypothesis of bihemispheric, asymmetric networks subserving memory processes and revealed both similarities in general and differences in the interactions between brain regions during episodic encoding and retrieval as well as working memory. Changes in memory function with ageing are evident from studies in old volunteers activating more brain regions compared to young volunteers and revealing more and stronger influences of prefrontal regions. We finally discuss the way in which the systems level models based on PET and fMRI results have implications for the understanding of the underlying neural network functioning of the brain.     

Brain imaging and memory systems in humans: the contribution of PET methods

The development of neuroimaging methods such as PET, has provided a new impulse to the study of the neural basis of cognitive functions, and has extended the field of inquiry from the analysis of the consequences of brain lesions to the functional investigations of brain activity, either in patients with selective neuropsychological deficits or in normal subjects engaged in cognitive tasks. Specific patterns of hypometabolism in neurological patients are associated with different profiles of memory deficits. [¹⁸F]FDG PET studies have confirmed the association of episodic memory with the structures of Papez's circuit and have shown correlations between short-term and semantic memory and the language areas. The identification of anatomo-functional networks involved in specific components of memory function in normal subjects is the aim of several PET activation studies. The results are in agreement with ‘neural network’ models of the neural basis of memory, as complex functions subserved by multiple interconnected cortical and subcortical structures.     

Integrating across neuroimaging modalities boosts prediction accuracy of cognitive ability

Variation in cognitive ability arises from subtle differences in underlying neural architecture. Understanding and predicting individual variability in cognition from the differences in brain networks requires harnessing the unique variance captured by different neuroimaging modalities. Here we adopted a multi-level machine learning approach that combines diffusion, functional, and structural MRI data from the Human Connectome Project (N = 1050) to provide unitary prediction models of various cognitive abilities: global cognitive function, fluid intelligence, crystallized intelligence, impulsivity, spatial orientation, verbal episodic memory and sustained attention. Out-of-sample predictions of each cognitive score were first generated using a sparsity-constrained principal component regression on individual neuroimaging modalities. These individual predictions were then aggregated and submitted to a LASSO estimator that removed redundant variability across channels. This stacked prediction led to a significant improvement in accuracy, relative to the best single modality predictions (approximately 1% to more than 3% boost in variance explained), across a majority of the cognitive abilities tested. Further analysis found that diffusion and brain surface properties contribute the most to the predictive power. Our findings establish a lower bound to predict individual differences in cognition using multiple neuroimaging measures of brain architecture, both structural and functional, quantify the relative predictive power of the different imaging modalities, and reveal how each modality provides unique and complementary information about individual differences in cognitive function.     

Integrated mechanisms of anticipation and rate-of-change computations in cortical circuits

Local neocortical circuits are characterized by stereotypical physiological and structural features that subserve generic computational operations. These basic computations of the cortical microcircuit emerge through the interplay of neuronal connectivity, cellular intrinsic properties, and synaptic plasticity dynamics. How these interacting mechanisms generate specific computational operations in the cortical circuit remains largely unknown. Here, we identify the neurophysiological basis of both the rate of change and anticipation computations on synaptic inputs in a cortical circuit. Through biophysically realistic computer simulations and neuronal recordings, we show that the rate-of-change computation is operated robustly in cortical networks through the combination of two ubiquitous brain mechanisms: short-term synaptic depression and spike-frequency adaptation. We then show how this rate-of-change circuit can be embedded in a convergently connected network to anticipate temporally incoming synaptic inputs, in quantitative agreement with experimental findings on anticipatory responses to moving stimuli in the primary visual cortex. Given the robustness of the mechanism and the widespread nature of the physiological machinery involved, we suggest that rate-of-change computation and temporal anticipation are principal, hard-wired functions of neural information processing in the cortical microcircuit.     

Physical models of neural networks

The state of art in computer modelling of neural networks with associative memory is reviewed. The available experimental data are considered on learning and memory of small neural systems, on isolated synapses and on molecular level. Computer simulations demonstrate that realistic models of neural ensembles exhibit properties which can be interpreted as image recognition, categorization, learning, prototype forming, etc. A bilayer model of associative neural network is proposed. One layer corresponds to the short-term memory, the other one to the long-term memory. Patterns are stored in terms of the synaptic strength matrix. We have studied the relaxational dynamics of neurons firing and suppression within the short-term memory layer under the influence of the long-term memory layer. The interaction among the layers has found to create a number of novel stable states which are not the learning patterns. These synthetic patterns may consist of elements belonging to different non-intersecting learning patterns. Within the framework of a hypothesis of selective and definite coding of images in brain one can interpret the observed effect as the "idea? generating" process.     

Aedes-AI: Neural network models of mosquito abundance

We present artificial neural networks as a feasible replacement for a mechanistic model of mosquito abundance. We develop a feed-forward neural network, a long short-term memory recurrent neural network, and a gated recurrent unit network. We evaluate the networks in their ability to replicate the spatiotemporal features of mosquito populations predicted by the mechanistic model, and discuss how augmenting the training data with time series that emphasize specific dynamical behaviors affects model performance. We conclude with an outlook on how such equation-free models may facilitate vector control or the estimation of disease risk at arbitrary spatial scales.     

Fading memory and kernel properties of generic cortical microcircuit models

It is quite difficult to construct circuits of spiking neurons that can carry out complex computational tasks. On the other hand even randomly connected circuits of spiking neurons can in principle be used for complex computational tasks such as time-warp invariant speech recognition. This is possible because such circuits have an inherent tendency to integrate incoming information in such a way that simple linear readouts can be trained to transform the current circuit activity into the target output for a very large number of computational tasks. Consequently we propose to analyze circuits of spiking neurons in terms of their roles as analog fading memory and non-linear kernels, rather than as implementations of specific computational operations and algorithms. This article is a sequel to [W. Maass, T. Natschl?ger, H. Markram, Real-time computing without stable states: a new framework for neural computation based on perturbations, Neural Comput. 14 (11) (2002) 2531-2560, Online available as #130 from: ], and contains new results about the performance of generic neural microcircuit models for the recognition of speech that is subject to linear and non-linear time-warps, as well as for computations on time-varying firing rates. These computations rely, apart from general properties of generic neural microcircuit models, just on capabilities of simple linear readouts trained by linear regression. This article also provides detailed data on the fading memory property of generic neural microcircuit models, and a quick review of other new results on the computational power of such circuits of spiking neurons.     

Neural economics and the biological substrates of valuation

A recent flurry of neuroimaging and decision-making experiments in humans, when combined with single-unit data from orbitofrontal cortex, suggests major additions to current models of reward processing. We review these data and models and use them to develop a specific computational relationship between the value of a predictor and the future rewards or punishments that it promises. The resulting computational model, the predictor-valuation model (PVM), is shown to anticipate a class of single-unit neural responses in orbitofrontal and striatal neurons. The model also suggests how neural responses in the orbitofrontal-striatal circuit may support the conversion of disparate types of future rewards into a kind of internal currency, that is, a common scale used to compare the valuation of future behavioral acts or stimuli.     

The ion channel inverse problem: neuroinformatics meets biophysics

Ion channels are the building blocks of the information processing capability of neurons: any realistic computational model of a neuron must include reliable and effective ion channel components. Sophisticated statistical and computational tools have been developed to study the ion channel structure–function relationship, but this work is rarely incorporated into the models used for single neurons or small networks. The disjunction is partly a matter of convention. Structure–function studies typically use a single Markov model for the whole channel whereas until recently whole-cell modeling software has focused on serial, independent, two-state subunits that can be represented by the Hodgkin–Huxley equations. More fundamentally, there is a difference in purpose that prevents models being easily reused. Biophysical models are typically developed to study one particular aspect of channel gating in detail, whereas neural modelers require broad coverage of the entire range of channel behavior that is often best achieved with approximate representations that omit structural features that cannot be adequately constrained. To bridge the gap so that more recent channel data can be used in neural models requires new computational infrastructure for bringing together diverse sources of data to arrive at best-fit models for whole-cell modeling. We review the current state of channel modeling and explore the developments needed for its conclusions to be integrated into whole-cell modeling.     

Population based models of cortical drug response: insights from anaesthesia

A great explanatory gap lies between the molecular pharmacology of psychoactive agents and the neurophysiological changes they induce, as recorded by neuroimaging modalities. Causally relating the cellular actions of psychoactive compounds to their influence on population activity is experimentally challenging. Recent developments in the dynamical modelling of neural tissue have attempted to span this explanatory gap between microscopic targets and their macroscopic neurophysiological effects via a range of biologically plausible dynamical models of cortical tissue. Such theoretical models allow exploration of neural dynamics, in particular their modification by drug action. The ability to theoretically bridge scales is due to a biologically plausible averaging of cortical tissue properties. In the resulting macroscopic neural field, individual neurons need not be explicitly represented (as in neural networks). The following paper aims to provide a non-technical introduction to the mean field population modelling of drug action and its recent successes in modelling anaesthesia.     

Integrating neuroscientific data across spatiotemporal scales

A major challenge confronting neuroscientists is associated with the multiple spatial and temporal scales of investigation of neural structure and function. I shall discuss the use of computational neural modeling as one method to bridge some of the different spatial and temporal levels. This approach will be illustrated using large-scale, neurobiologically realistic network models of auditory and visual pattern recognition that relate neuronal dynamics to fMRI data. It will be demonstrated that the models are capable of exhibiting the salient features of both electrophysiological neuronal activities and fMRI values that are in agreement with empirically observed data.     

From cognitive to neural models of working memory

Working memory refers to the temporary retention of information that was just experienced or just retrieved from long-term memory but no longer exists in the external environment. These internal representations are short-lived, but can be stored for longer periods of time through active maintenance or rehearsal strategies, and can be subjected to various operations that manipulate the information in such a way that makes it useful for goal-directed behaviour. Empirical studies of working memory using neuroscientific techniques, such as neuronal recordings in monkeys or functional neuroimaging in humans, have advanced our knowledge of the underlying neural mechanisms of working memory. This rich dataset can be reconciled with behavioural findings derived from investigating the cognitive mechanisms underlying working memory. In this paper, I review the progress that has been made towards this effort by illustrating how investigations of the neural mechanisms underlying working memory can be influenced by cognitive models and, in turn, how cognitive models can be shaped and modified by neuroscientific data. One conclusion that arises from this research is that working memory can be viewed as neither a unitary nor a dedicated system. A network of brain regions, including the prefrontal cortex (PFC), is critical for the active maintenance of internal representations that are necessary for goal-directed behaviour. Thus, working memory is not localized to a single brain region but probably is an emergent property of the functional interactions between the PFC and the rest of the brain.     

Modeling regulatory networks at Virginia Tech

The life of a cell is governed by the physicochemical properties of a complex network of interacting macromolecules (primarily genes and proteins). Hence, a full scientific understanding of and rational engineering approach to cell physiology require accurate mathematical models of the spatial and temporal dynamics of these macromolecular assemblies, especially the networks involved in integrating signals and regulating cellular responses. The Virginia Tech Consortium is involved in three specific goals of DARPA's computational biology program (Bio-COMP): to create effective software tools for modeling gene-protein-metabolite networks, to employ these tools in creating a new generation of realistic models, and to test and refine these models by well-conceived experimental studies. The special emphasis of this group is to understand the mechanisms of cell cycle control in eukaryotes (yeast cells and frog eggs). The software tools developed at Virginia Tech are designed to meet general requirements of modeling regulatory networks and are collected in a problem-solving environment called JigCell.     

A hybrid model for the neural representation of complex mental processing in the human brain

In the present conceptual review several theoretical and empirical sources of information were integrated, and a hybrid model of the neural representation of complex mental processing in the human brain was proposed. Based on empirical evidence for strategy-related and inter-individually different task-related brain activation networks, and further based on empirical evidence for a remarkable overlap of fronto-parietal activation networks across different complex mental processes, it was concluded by the author that there might be innate and modular organized neuro-developmental starting regions, for example, in intra-parietal, and both medial and middle frontal brain regions, from which the neural organization of different kinds of complex mental processes emerge differently during individually shaped learning histories. Thus, the here proposed model provides a hybrid of both massive modular and holistic concepts of idiosyncratic brain physiological elaboration of complex mental processing. It is further concluded that 3-D information, obtained by respective methodological approaches, are not appropriate to identify the non-linear spatio-temporal dynamics of complex mental process-related brain activity in a sufficient way. How different participating network parts communicate with each other seems to be an indispensable aspect, which has to be considered in particular to improve our understanding of the neural organization of complex cognition.     

Testing neural network models of memory with behavioral experiments

In recent years, a number of computational neural networks have been proposed aimed at describing memory functions associated with different subregions of the hippocampus, namely dentate gyrus, CA3 and CA1. Recent evidence suggests that indeed specific subregions of the hippocampus may subserve different computational functions, such as spatial and temporal pattern separation, short-term or working memory, pattern association, and temporal pattern completion.     

Functional imaging of the auditory processing applied to speech sounds

In this paper, we describe domain-general auditory processes that we believe are prerequisite to the linguistic analysis of speech. We discuss biological evidence for these processes and how they might relate to processes that are specific to human speech and language. We begin with a brief review of (i) the anatomy of the auditory system and (ii) the essential properties of speech sounds. Section 4 describes the general auditory mechanisms that we believe are applied to all communication sounds, and how functional neuroimaging is being used to map the brain networks associated with domain-general auditory processing. Section 5 discusses recent neuroimaging studies that explore where such general processes give way to those that are specific to human speech and language.     

A linear model for characterization of synchronization frequencies of neural networks

The synchronization frequency of neural networks and its dynamics have important roles in deciphering the working mechanisms of the brain. It has been widely recognized that the properties of functional network synchronization and its dynamics are jointly determined by network topology, network connection strength, i.e., the connection strength of different edges in the network, and external input signals, among other factors. However, mathematical and computational characterization of the relationships between network synchronization frequency and these three important factors are still lacking. This paper presents a novel computational simulation framework to quantitatively characterize the relationships between neural network synchronization frequency and network attributes and input signals. Specifically, we constructed a series of neural networks including simulated small-world networks, real functional working memory network derived from functional magnetic resonance imaging, and real large-scale structural brain networks derived from diffusion tensor imaging, and performed synchronization simulations on these networks via the Izhikevich neuron spiking model. Our experiments demonstrate that both of the network synchronization strength and synchronization frequency change according to the combination of input signal frequency and network self-synchronization frequency. In particular, our extensive experiments show that the network synchronization frequency can be represented via a linear combination of the network self-synchronization frequency and the input signal frequency. This finding could be attributed to an intrinsically-preserved principle in different types of neural systems, offering novel insights into the working mechanism of neural systems.