Fusing EEG and fMRI based on a bottom-up model: inferring activation and effective connectivity in neural masses

The elucidation of the complex machinery used by the human brain to segregate and integrate information while performing high cognitive functions is a subject of imminent future consequences. The most significant contributions to date in this field, known as cognitive neuroscience, have been achieved by using innovative neuroimaging techniques, such as electroencephalogram (EEG) and functional magnetic resonance imaging (fMRI), which measure variations in both the time and the space of some interpretable physical magnitudes. Extraordinary maps of cerebral activation involving function-restricted brain areas, as well as graphs of the functional connectivity between them, have been obtained from EEG and fMRI data by solving some spatio-temporal inverse problems, which constitutes a top-down approach. However, in many cases, a natural bridge between these maps/graphs and the causal physiological processes is lacking, leading to some misunderstandings in their interpretation. Recent advances in the comprehension of the underlying physiological mechanisms associated with different cerebral scales have provided researchers with an excellent scenario to develop sophisticated biophysical models that permit an integration of these neuroimage modalities, which must share a common aetiology. This paper proposes a bottom-up approach, involving physiological parameters in a specific mesoscopic dynamic equations system. Further observation equations encapsulating the relationship between the mesostates and the EEG/fMRI data are obtained on the basis of the physical foundations of these techniques. A methodology for the estimation of parameters from fused EEG/fMRI data is also presented. In this context, the concepts of activation and effective connectivity are carefully revised. This new approach permits us to examine and discuss some future prospects for the integration of multimodal neuroimages.     

Large-Scale Functional Networks Identified from Resting-State EEG Using Spatial ICA

Several methods have been applied to EEG or MEG signals to detect functional networks. In recent works using MEG/EEG and fMRI data, temporal ICA analysis has been used to extract spatial maps of resting-state networks with or without an atlas-based parcellation of the cortex. Since the links between the fMRI signal and the electromagnetic signals are not fully established, and to avoid any bias, we examined whether EEG alone was able to derive the spatial distribution and temporal characteristics of functional networks. To do so, we propose a two-step original method: 1) An individual multi-frequency data analysis including EEG-based source localisation and spatial independent component analysis, which allowed us to characterize the resting-state networks. 2) A group-level analysis involving a hierarchical clustering procedure to identify reproducible large-scale networks across the population. Compared with large-scale resting-state networks obtained with fMRI, the proposed EEG-based analysis revealed smaller independent networks thanks to the high temporal resolution of EEG, hence hierarchical organization of networks. The comparison showed a substantial overlap between EEG and fMRI networks in motor, premotor, sensory, frontal, and parietal areas. However, there were mismatches between EEG-based and fMRI-based networks in temporal areas, presumably resulting from a poor sensitivity of fMRI in these regions or artefacts in the EEG signals. The proposed method opens the way for studying the high temporal dynamics of networks at the source level thanks to the high temporal resolution of EEG. It would then become possible to study detailed measures of the dynamics of connectivity.     

Double‐wavelet transform for multisubject task‐induced functional magnetic resonance imaging data

The goal of this article is to model multisubject task‐induced functional magnetic resonance imaging (fMRI) response among predefined regions of interest (ROIs) of the human brain. Conventional approaches to fMRI analysis only take into account temporal correlations, but do not rigorously model the underlying spatial correlation due to the complexity of estimating and inverting the high dimensional spatio‐temporal covariance matrix. Other spatio‐temporal model approaches estimate the covariance matrix with the assumption of stationary time series, which is not always feasible. To address these limitations, we propose a double‐wavelet approach for modeling the spatio‐temporal brain process. Working with wavelet coefficients simplifies temporal and spatial covariance structure because under regularity conditions, wavelet coefficients are approximately uncorrelated. Different wavelet functions were used to capture different correlation structures in the spatio‐temporal model. The main advantages of the wavelet approach are that it is scalable and that it deals with nonstationarity in brain signals. Simulation studies showed that our method could reduce false‐positive and false‐negative rates by taking into account spatial and temporal correlations simultaneously. We also applied our method to fMRI data to study activation in prespecified ROIs in the prefontal cortex. Data analysis showed that the result using the double‐wavelet approach was more consistent than the conventional approach when sample size decreased.     

Multimodal functional network connectivity: an EEG-fMRI fusion in network space

EEG and fMRI recordings measure the functional activity of multiple coherent networks distributed in the cerebral cortex. Identifying network interaction from the complementary neuroelectric and hemodynamic signals may help to explain the complex relationships between different brain regions. In this paper, multimodal functional network connectivity (mFNC) is proposed for the fusion of EEG and fMRI in network space. First, functional networks (FNs) are extracted using spatial independent component analysis (ICA) in each modality separately. Then the interactions among FNs in each modality are explored by Granger causality analysis (GCA). Finally, fMRI FNs are matched to EEG FNs in the spatial domain using network-based source imaging (NESOI). Investigations of both synthetic and real data demonstrate that mFNC has the potential to reveal the underlying neural networks of each modality separately and in their combination. With mFNC, comprehensive relationships among FNs might be unveiled for the deep exploration of neural activities and metabolic responses in a specific task or neurological state.     

The brain timewise: how timing shapes and supports brain function

We discuss the importance of timing in brain function: how temporal dynamics of the world has left its traces in the brain during evolution and how we can monitor the dynamics of the human brain with non-invasive measurements. Accurate timing is important for the interplay of neurons, neuronal circuitries, brain areas and human individuals. In the human brain, multiple temporal integration windows are hierarchically organized, with temporal scales ranging from microseconds to tens and hundreds of milliseconds for perceptual, motor and cognitive functions, and up to minutes, hours and even months for hormonal and mood changes. Accurate timing is impaired in several brain diseases. From the current repertoire of non-invasive brain imaging methods, only magnetoencephalography (MEG) and scalp electroencephalography (EEG) provide millisecond time-resolution; our focus in this paper is on MEG. Since the introduction of high-density whole-scalp MEG/EEG coverage in the 1990s, the instrumentation has not changed drastically; yet, novel data analyses are advancing the field rapidly by shifting the focus from the mere pinpointing of activity hotspots to seeking stimulus- or task-specific information and to characterizing functional networks. During the next decades, we can expect increased spatial resolution and accuracy of the time-resolved brain imaging and better understanding of brain function, especially its temporal constraints, with the development of novel instrumentation and finer-grained, physiologically inspired generative models of local and network activity. Merging both spatial and temporal information with increasing accuracy and carrying out recordings in naturalistic conditions, including social interaction, will bring much new information about human brain function.     

Unveiling functions of the visual cortex using task-specific deep neural networks

The human visual cortex enables visual perception through a cascade of hierarchical computations in cortical regions with distinct functionalities. Here, we introduce an AI-driven approach to discover the functional mapping of the visual cortex. We related human brain responses to scene images measured with functional MRI (fMRI) systematically to a diverse set of deep neural networks (DNNs) optimized to perform different scene perception tasks. We found a structured mapping between DNN tasks and brain regions along the ventral and dorsal visual streams. Low-level visual tasks mapped onto early brain regions, 3-dimensional scene perception tasks mapped onto the dorsal stream, and semantic tasks mapped onto the ventral stream. This mapping was of high fidelity, with more than 60% of the explainable variance in nine key regions being explained. Together, our results provide a novel functional mapping of the human visual cortex and demonstrate the power of the computational approach.     

Resting-State Temporal Synchronization Networks Emerge from Connectivity Topology and Heterogeneity

Spatial patterns of coherent activity across different brain areas have been identified during the resting-state fluctuations of the brain. However, recent studies indicate that resting-state activity is not stationary, but shows complex temporal dynamics. We were interested in the spatiotemporal dynamics of the phase interactions among resting-state fMRI BOLD signals from human subjects. We found that the global phase synchrony of the BOLD signals evolves on a characteristic ultra-slow (<0.01Hz) time scale, and that its temporal variations reflect the transient formation and dissolution of multiple communities of synchronized brain regions. Synchronized communities reoccurred intermittently in time and across scanning sessions. We found that the synchronization communities relate to previously defined functional networks known to be engaged in sensory-motor or cognitive function, called resting-state networks (RSNs), including the default mode network, the somato-motor network, the visual network, the auditory network, the cognitive control networks, the self-referential network, and combinations of these and other RSNs. We studied the mechanism originating the observed spatiotemporal synchronization dynamics by using a network model of phase oscillators connected through the brain’s anatomical connectivity estimated using diffusion imaging human data. The model consistently approximates the temporal and spatial synchronization patterns of the empirical data, and reveals that multiple clusters that transiently synchronize and desynchronize emerge from the complex topology of anatomical connections, provided that oscillators are heterogeneous.     

Extracting Visual Evoked Potentials from EEG Data Recorded During fMRI-guided Transcranial Magnetic Stimulation

Transcranial Magnetic Stimulation (TMS) is an effective method for establishing a causal link between a cortical area and cognitive/neurophysiological effects. Specifically, by creating a transient interference with the normal activity of a target region and measuring changes in an electrophysiological signal, we can establish a causal link between the stimulated brain area or network and the electrophysiological signal that we record. If target brain areas are functionally defined with prior fMRI scan, TMS could be used to link the fMRI activations with evoked potentials recorded. However, conducting such experiments presents significant technical challenges given the high amplitude artifacts introduced into the EEG signal by the magnetic pulse, and the difficulty to successfully target areas that were functionally defined by fMRI. Here we describe a methodology for combining these three common tools: TMS, EEG, and fMRI. We explain how to guide the stimulator''s coil to the desired target area using anatomical or functional MRI data, how to record EEG during concurrent TMS, how to design an ERP study suitable for EEG-TMS combination and how to extract reliable ERP from the recorded data. We will provide representative results from a previously published study, in which fMRI-guided TMS was used concurrently with EEG to show that the face-selective N1 and the body-selective N1 component of the ERP are associated with distinct neural networks in extrastriate cortex. This method allows us to combine the high spatial resolution of fMRI with the high temporal resolution of TMS and EEG and therefore obtain a comprehensive understanding of the neural basis of various cognitive processes.     

Fast two-photon in vivo imaging with three-dimensional random-access scanning in large tissue volumes

The understanding of brain computations requires methods that read out neural activity on different spatial and temporal scales. Following signal propagation and integration across a neuron and recording the concerted activity of hundreds of neurons pose distinct challenges, and the design of imaging systems has been mostly focused on tackling one of the two operations. We developed a high-resolution, acousto-optic two-photon microscope with continuous three-dimensional (3D) trajectory and random-access scanning modes that reaches near-cubic-millimeter scan range and can be adapted to imaging different spatial scales. We performed 3D calcium imaging of action potential backpropagation and dendritic spike forward propagation at sub-millisecond temporal resolution in mouse brain slices. We also performed volumetric random-access scanning calcium imaging of spontaneous and visual stimulation-evoked activity in hundreds of neurons of the mouse visual cortex in vivo. These experiments demonstrate the subcellular and network-scale imaging capabilities of our system.     

Resting State Networks' Corticotopy: The Dual Intertwined Rings Architecture

How does the brain integrate multiple sources of information to support normal sensorimotor and cognitive functions? To investigate this question we present an overall brain architecture (called “the dual intertwined rings architecture”) that relates the functional specialization of cortical networks to their spatial distribution over the cerebral cortex (or “corticotopy”). Recent results suggest that the resting state networks (RSNs) are organized into two large families: 1) a sensorimotor family that includes visual, somatic, and auditory areas and 2) a large association family that comprises parietal, temporal, and frontal regions and also includes the default mode network. We used two large databases of resting state fMRI data, from which we extracted 32 robust RSNs. We estimated: (1) the RSN functional roles by using a projection of the results on task based networks (TBNs) as referenced in large databases of fMRI activation studies; and (2) relationship of the RSNs with the Brodmann Areas. In both classifications, the 32 RSNs are organized into a remarkable architecture of two intertwined rings per hemisphere and so four rings linked by homotopic connections. The first ring forms a continuous ensemble and includes visual, somatic, and auditory cortices, with interspersed bimodal cortices (auditory-visual, visual-somatic and auditory-somatic, abbreviated as VSA ring). The second ring integrates distant parietal, temporal and frontal regions (PTF ring) through a network of association fiber tracts which closes the ring anatomically and ensures a functional continuity within the ring. The PTF ring relates association cortices specialized in attention, language and working memory, to the networks involved in motivation and biological regulation and rhythms. This “dual intertwined architecture” suggests a dual integrative process: the VSA ring performs fast real-time multimodal integration of sensorimotor information whereas the PTF ring performs multi-temporal integration (i.e., relates past, present, and future representations at different temporal scales).     

Brain connectivity at different time-scales measured with EEG

We present an overview of different methods for decomposing a multichannel spontaneous electroencephalogram (EEG) into sets of temporal patterns and topographic distributions. All of the methods presented here consider the scalp electric field as the basic analysis entity in space. In time, the resolution of the methods is between milliseconds (time-domain analysis), subseconds (time- and frequency-domain analysis) and seconds (frequency-domain analysis). For any of these methods, we show that large parts of the data can be explained by a small number of topographic distributions. Physically, this implies that the brain regions that generated one of those topographies must have been active with a common phase. If several brain regions are producing EEG signals at the same time and frequency, they have a strong tendency to do this in a synchronized mode. This view is illustrated by several examples (including combined EEG and functional magnetic resonance imaging (fMRI)) and a selective review of the literature. The findings are discussed in terms of short-lasting binding between different brain regions through synchronized oscillations, which could constitute a mechanism to form transient, functional neurocognitive networks.     

Human gamma oscillations during slow wave sleep

Neocortical local field potentials have shown that gamma oscillations occur spontaneously during slow-wave sleep (SWS). At the macroscopic EEG level in the human brain, no evidences were reported so far. In this study, by using simultaneous scalp and intracranial EEG recordings in 20 epileptic subjects, we examined gamma oscillations in cerebral cortex during SWS. We report that gamma oscillations in low (30-50 Hz) and high (60-120 Hz) frequency bands recurrently emerged in all investigated regions and their amplitudes coincided with specific phases of the cortical slow wave. In most of the cases, multiple oscillatory bursts in different frequency bands from 30 to 120 Hz were correlated with positive peaks of scalp slow waves ("IN-phase" pattern), confirming previous animal findings. In addition, we report another gamma pattern that appears preferentially during the negative phase of the slow wave ("ANTI-phase" pattern). This new pattern presented dominant peaks in the high gamma range and was preferentially expressed in the temporal cortex. Finally, we found that the spatial coherence between cortical sites exhibiting gamma activities was local and fell off quickly when computed between distant sites. Overall, these results provide the first human evidences that gamma oscillations can be observed in macroscopic EEG recordings during sleep. They support the concept that these high-frequency activities might be associated with phasic increases of neural activity during slow oscillations. Such patterned activity in the sleeping brain could play a role in off-line processing of cortical networks.     

A Novel Method for the Topographic Analysis of Neural Activity Reveals Formation and Dissolution of ‘Dynamic Cell Assemblies’

The study of synchronous oscillations in neural systems is a very active area of research. However, cognitive function may depend more crucially upon a dynamic alternation between synchronous and desynchronous activity rather than synchronous behaviour per se. The principle aim of this study is to develop and validate a novel method of quantifying this complex process. The method permits a direct mapping of phase synchronous dynamics and desynchronizing bursts in the spatial and temporal domains. Two data sets are analyzed: Numeric data from a model of a sparsely coupled neural cell assembly and experimental data consisting of scalp-recorded EEG from 40 human subjects. In the numeric data, the approach enables the demonstration of complex relationships between cluster size and temporal duration that cannot be detected with other methods. Dynamic patterns of phase-clustering and desynchronization are also demonstrated in the experimental data. It is further shown that in a significant proportion of the recordings, the pattern of dynamics exhibits nonlinear structure. We argue that this procedure provides a 'natural partitioning' of ongoing brain dynamics into topographically distinct synchronous epochs which may be integral to the brain's adaptive function. In particular, the character of transitions between consecutive synchronous epochs may reflect important aspects of information processing and cognitive flexibility.     

The Perception of Dynamic and Static Facial Expressions of Happiness and Disgust Investigated by ERPs and fMRI Constrained Source Analysis

A recent functional magnetic resonance imaging (fMRI) study by our group demonstrated that dynamic emotional faces are more accurately recognized and evoked more widespread patterns of hemodynamic brain responses than static emotional faces. Based on this experimental design, the present study aimed at investigating the spatio-temporal processing of static and dynamic emotional facial expressions in 19 healthy women by means of multi-channel electroencephalography (EEG), event-related potentials (ERP) and fMRI-constrained regional source analyses. ERP analysis showed an increased amplitude of the LPP (late posterior positivity) over centro-parietal regions for static facial expressions of disgust compared to neutral faces. In addition, the LPP was more widespread and temporally prolonged for dynamic compared to static faces of disgust and happiness. fMRI constrained source analysis on static emotional face stimuli indicated the spatio-temporal modulation of predominantly posterior regional brain activation related to the visual processing stream for both emotional valences when compared to the neutral condition in the fusiform gyrus. The spatio-temporal processing of dynamic stimuli yielded enhanced source activity for emotional compared to neutral conditions in temporal (e.g., fusiform gyrus), and frontal regions (e.g., ventromedial prefrontal cortex, medial and inferior frontal cortex) in early and again in later time windows. The present data support the view that dynamic facial displays trigger more information reflected in complex neural networks, in particular because of their changing features potentially triggering sustained activation related to a continuing evaluation of those faces. A combined fMRI and EEG approach thus provides an advanced insight to the spatio-temporal characteristics of emotional face processing, by also revealing additional neural generators, not identifiable by the only use of an fMRI approach.     

Enhancing oscillations in intracranial electrophysiological recordings with data-driven spatial filters

In invasive electrophysiological recordings, a variety of neural oscillations can be detected across the cortex, with overlap in space and time. This overlap complicates measurement of neural oscillations using standard referencing schemes, like common average or bipolar referencing. Here, we illustrate the effects of spatial mixing on measuring neural oscillations in invasive electrophysiological recordings and demonstrate the benefits of using data-driven referencing schemes in order to improve measurement of neural oscillations. We discuss referencing as the application of a spatial filter. Spatio-spectral decomposition is used to estimate data-driven spatial filters, a computationally fast method which specifically enhances signal-to-noise ratio for oscillations in a frequency band of interest. We show that application of these data-driven spatial filters has benefits for data exploration, investigation of temporal dynamics and assessment of peak frequencies of neural oscillations. We demonstrate multiple use cases, exploring between-participant variability in presence of oscillations, spatial spread and waveform shape of different rhythms as well as narrowband noise removal with the aid of spatial filters. We find high between-participant variability in the presence of neural oscillations, a large variation in spatial spread of individual rhythms and many non-sinusoidal rhythms across the cortex. Improved measurement of cortical rhythms will yield better conditions for establishing links between cortical activity and behavior, as well as bridging scales between the invasive intracranial measurements and noninvasive macroscale scalp measurements.     

脑网络：从脑结构到脑功能

人脑是自然界中最复杂的系统之一,不同的功能区域相互作用、互相协调,共同构成一个网络来发挥其功能。人脑是一个复杂的网络,具有高效的“小世界”拓扑属性。本文从脑结构到脑功能方面介绍了从不同模态影像学数据构造脑网络的主要进展,并探讨不同的脑疾病患者脑网络拓扑结构是否发生了异常,以及这些异常特征能否用来进行疾病分类,最后对本领域未来的研究做了简单的展望。     

Investigations into resting-state connectivity using independent component analysis

Inferring resting-state connectivity patterns from functional magnetic resonance imaging (fMRI) data is a challenging task for any analytical technique. In this paper, we review a probabilistic independent component analysis (PICA) approach, optimized for the analysis of fMRI data, and discuss the role which this exploratory technique can take in scientific investigations into the structure of these effects. We apply PICA to fMRI data acquired at rest, in order to characterize the spatio-temporal structure of such data, and demonstrate that this is an effective and robust tool for the identification of low-frequency resting-state patterns from data acquired at various different spatial and temporal resolutions. We show that these networks exhibit high spatial consistency across subjects and closely resemble discrete cortical functional networks such as visual cortical areas or sensory-motor cortex.     

Task-Driven Activity Reduces the Cortical Activity Space of the Brain: Experiment and Whole-Brain Modeling

How a stimulus or a task alters the spontaneous dynamics of the brain remains a fundamental open question in neuroscience. One of the most robust hallmarks of task/stimulus-driven brain dynamics is the decrease of variability with respect to the spontaneous level, an effect seen across multiple experimental conditions and in brain signals observed at different spatiotemporal scales. Recently, it was observed that the trial-to-trial variability and temporal variance of functional magnetic resonance imaging (fMRI) signals decrease in the task-driven activity. Here we examined the dynamics of a large-scale model of the human cortex to provide a mechanistic understanding of these observations. The model allows computing the statistics of synaptic activity in the spontaneous condition and in putative tasks determined by external inputs to a given subset of brain regions. We demonstrated that external inputs decrease the variance, increase the covariances, and decrease the autocovariance of synaptic activity as a consequence of single node and large-scale network dynamics. Altogether, these changes in network statistics imply a reduction of entropy, meaning that the spontaneous synaptic activity outlines a larger multidimensional activity space than does the task-driven activity. We tested this model’s prediction on fMRI signals from healthy humans acquired during rest and task conditions and found a significant decrease of entropy in the stimulus-driven activity. Altogether, our study proposes a mechanism for increasing the information capacity of brain networks by enlarging the volume of possible activity configurations at rest and reliably settling into a confined stimulus-driven state to allow better transmission of stimulus-related information.