Encoding brain network response to free viewing of videos

A challenging goal for cognitive neuroscience researchers is to determine how mental representations are mapped onto the patterns of neural activity. To address this problem, functional magnetic resonance imaging (fMRI) researchers have developed a large number of encoding and decoding methods. However, previous studies typically used rather limited stimuli representation, like semantic labels and Wavelet Gabor filters, and largely focused on voxel-based brain patterns. Here, we present a new fMRI encoding model to predict the human brain’s responses to free viewing of video clips which aims to deal with this limitation. In this model, we represent the stimuli using a variety of representative visual features in the computer vision community, which can describe the global color distribution, local shape and spatial information and motion information contained in videos, and apply the functional connectivity to model the brain’s activity pattern evoked by these video clips. Our experimental results demonstrate that brain network responses during free viewing of videos can be robustly and accurately predicted across subjects by using visual features. Our study suggests the feasibility of exploring cognitive neuroscience studies by computational image/video analysis and provides a novel concept of using the brain encoding as a test-bed for evaluating visual feature extraction.     

High Density Event-related Potential Data Acquisition in Cognitive Neuroscience

Functional magnetic resonance imaging (fMRI) is currently the standard method of evaluating brain function in the field of Cognitive Neuroscience, in part because fMRI data acquisition and analysis techniques are readily available. Because fMRI has excellent spatial resolution but poor temporal resolution, this method can only be used to identify the spatial location of brain activity associated with a given cognitive process (and reveals virtually nothing about the time course of brain activity). By contrast, event-related potential (ERP) recording, a method that is used much less frequently than fMRI, has excellent temporal resolution and thus can track rapid temporal modulations in neural activity. Unfortunately, ERPs are under utilized in Cognitive Neuroscience because data acquisition techniques are not readily available and low density ERP recording has poor spatial resolution. In an effort to foster the increased use of ERPs in Cognitive Neuroscience, the present article details key techniques involved in high density ERP data acquisition. Critically, high density ERPs offer the promise of excellent temporal resolution and good spatial resolution (or excellent spatial resolution if coupled with fMRI), which is necessary to capture the spatial-temporal dynamics of human brain function.Download video file.^{(101M, mp4)}     

Extracting Visual Evoked Potentials from EEG Data Recorded During fMRI-guided Transcranial Magnetic Stimulation

Transcranial Magnetic Stimulation (TMS) is an effective method for establishing a causal link between a cortical area and cognitive/neurophysiological effects. Specifically, by creating a transient interference with the normal activity of a target region and measuring changes in an electrophysiological signal, we can establish a causal link between the stimulated brain area or network and the electrophysiological signal that we record. If target brain areas are functionally defined with prior fMRI scan, TMS could be used to link the fMRI activations with evoked potentials recorded. However, conducting such experiments presents significant technical challenges given the high amplitude artifacts introduced into the EEG signal by the magnetic pulse, and the difficulty to successfully target areas that were functionally defined by fMRI. Here we describe a methodology for combining these three common tools: TMS, EEG, and fMRI. We explain how to guide the stimulator''s coil to the desired target area using anatomical or functional MRI data, how to record EEG during concurrent TMS, how to design an ERP study suitable for EEG-TMS combination and how to extract reliable ERP from the recorded data. We will provide representative results from a previously published study, in which fMRI-guided TMS was used concurrently with EEG to show that the face-selective N1 and the body-selective N1 component of the ERP are associated with distinct neural networks in extrastriate cortex. This method allows us to combine the high spatial resolution of fMRI with the high temporal resolution of TMS and EEG and therefore obtain a comprehensive understanding of the neural basis of various cognitive processes.     

Regional variation in neurovascular coupling and why we still lack a Rosetta Stone

Functional magnetic resonance imaging (fMRI) is the dominant tool in cognitive neuroscience although its relation to underlying neural activity, particularly in the human brain, remains largely unknown. A major research goal, therefore, has been to uncover a ‘Rosetta Stone’ providing direct translation between the blood oxygen level-dependent (BOLD) signal, the local field potential and single-neuron activity. Here, I evaluate the proposal that BOLD signal changes equate to changes in gamma-band activity, which in turn may partially relate to the spiking activity of neurons. While there is some support for this idea in sensory cortices, findings in deeper brain structures like the hippocampus instead suggest both regional and frequency-wise differences. Relatedly, I consider four important factors in linking fMRI to neural activity: interpretation of correlations between these signals, regional variability in local vasculature, distributed neural coding schemes and varying fMRI signal quality. Novel analytic fMRI techniques, such as multivariate pattern analysis (MVPA), employ the distributed patterns of voxels across a brain region to make inferences about information content rather than whether a small number of voxels go up or down relative to baseline in response to a stimulus. Although unlikely to provide a Rosetta Stone, MVPA, therefore, may represent one possible means forward for better linking BOLD signal changes to the information coded by underlying neural activity.This article is part of the theme issue ‘Key relationships between non-invasive functional neuroimaging and the underlying neuronal activity’.     

脑源定位技术的精度评估及其在实际中的应用

脑源定位技术旨在通过头皮表面的脑电、脑磁信号来识别大脑内的神经活动源，是研究大脑皮层神经活动、认知过程和病理功能的基础。其毫秒级的时间分辨率可以有效弥补功能核磁共振在低时间分辨率方面的不足。然而，理论分析层面中逆问题的不适定性，以及实践操作层面上不同的记录方式、电极数量和头模型构建等过程带来的误差，给脑源定位的准确性带来极大挑战，也在一定程度上限制了脑源定位方法在神经科学和心理学研究以及临床诊断治疗中的实际应用。因此，理论分析和实践操作层面中的精度评估在脑源定位方法的实际使用中至关重要。针对以上问题，本文在对现有脑源定位方法介绍的基础上，着重分析了脑源定位技术的精度评估方法以及其在基础研究和临床诊断治疗中的实际应用。具体地，本文在理论分析中总结了基于空间分辨率、基于点扩散以及串扰函数的评估方法对于不同脑源定位方法中源的重叠程度和其他源对目标源的影响；在实践操作中介绍了记录方式、电极数量和密度、头部容积传导模型等因素对源定位精度的影响；进一步介绍了脑源定位技术在时频分析、连通性分析中的应用，以及其在临床中的应用，包括癫痫、注意缺陷与多动障碍等脑部疾病。     

Task-Driven Activity Reduces the Cortical Activity Space of the Brain: Experiment and Whole-Brain Modeling

How a stimulus or a task alters the spontaneous dynamics of the brain remains a fundamental open question in neuroscience. One of the most robust hallmarks of task/stimulus-driven brain dynamics is the decrease of variability with respect to the spontaneous level, an effect seen across multiple experimental conditions and in brain signals observed at different spatiotemporal scales. Recently, it was observed that the trial-to-trial variability and temporal variance of functional magnetic resonance imaging (fMRI) signals decrease in the task-driven activity. Here we examined the dynamics of a large-scale model of the human cortex to provide a mechanistic understanding of these observations. The model allows computing the statistics of synaptic activity in the spontaneous condition and in putative tasks determined by external inputs to a given subset of brain regions. We demonstrated that external inputs decrease the variance, increase the covariances, and decrease the autocovariance of synaptic activity as a consequence of single node and large-scale network dynamics. Altogether, these changes in network statistics imply a reduction of entropy, meaning that the spontaneous synaptic activity outlines a larger multidimensional activity space than does the task-driven activity. We tested this model’s prediction on fMRI signals from healthy humans acquired during rest and task conditions and found a significant decrease of entropy in the stimulus-driven activity. Altogether, our study proposes a mechanism for increasing the information capacity of brain networks by enlarging the volume of possible activity configurations at rest and reliably settling into a confined stimulus-driven state to allow better transmission of stimulus-related information.     

Self-regulation of local brain activity using real-time functional magnetic resonance imaging (fMRI)

Functional magnetic resonance imaging (fMRI) measures the blood oxygen level-dependent (BOLD) signal related to neuronal activity. So far, this technique has been limited by time-consuming data analysis impeding on-line analysis. In particular, no brain-computer interface (BCI) was available which provided on-line feedback to learn physiological self-regulation of the BOLD signal. Recently, studies have shown that fMRI feedback is feasible and facilitates voluntary control of brain activity. Here we review these studies to make the fMRI feedback methodology accessible to a broader scientific community such as researchers concerned with functional brain imaging and the neurobiology of learning. Methodological and conceptual limitations were substantially reduced by artefact control, sensitivity improvements, real-time algorithms, and adapted experimental designs. Physiological self-regulation of the local BOLD response is a new paradigm for cognitive neuroscience to study brain plasticity and the functional relevance of regulated brain areas by modification of behaviour. Voluntary control of abnormal activity in circumscribed brain areas may even be applied as psychophysiological treatment.     

Current state-of-the-art of auditory functional MRI (fMRI) on zebra finches: Technique and scientific achievements

Songbirds provide an excellent model system exhibiting vocal learning associated with an extreme brain plasticity linked to quantifiable behavioral changes. This animal model has thus far been intensively studied using electrophysiological, histological and molecular mapping techniques. However, these approaches do not provide a global view of the brain and/or do not allow repeated measures, which are necessary to establish correlations between alterations in neural substrate and behavior. In contrast, functional Magnetic Resonance Imaging (fMRI) is a non-invasive in vivo technique which allows one (i) to study brain function in the same subject over time, and (ii) to address the entire brain at once. During the last decades, fMRI has become one of the most popular neuroimaging techniques in cognitive neuroscience for the study of brain activity during various tasks ranging from simple sensory-motor to highly cognitive tasks. By alternating various stimulation periods with resting periods during scanning, resting and task-specific regional brain activity can be determined with this technique. Despite its obvious benefits, fMRI has, until now, only been sparsely used to study cognition in non-human species such as songbirds. The Bio-Imaging Lab (University of Antwerp, Belgium) was the first to implement Blood Oxygen Level Dependent (BOLD) fMRI in songbirds – and in particular zebra finches – for the visualization of sound perception and processing in auditory and song control brain regions. The present article provides an overview of the establishment and optimization of this technique in our laboratory and of the resulting scientific findings. The introduction of fMRI in songbirds has opened new research avenues that permit experimental analysis of complex sensorimotor and cognitive processes underlying vocal communication in this animal model.     

Spontaneous Decoding of the Timing and Content of Human Object Perception from Cortical Surface Recordings Reveals Complementary Information in the Event-Related Potential and Broadband Spectral Change

The link between object perception and neural activity in visual cortical areas is a problem of fundamental importance in neuroscience. Here we show that electrical potentials from the ventral temporal cortical surface in humans contain sufficient information for spontaneous and near-instantaneous identification of a subject’s perceptual state. Electrocorticographic (ECoG) arrays were placed on the subtemporal cortical surface of seven epilepsy patients. Grayscale images of faces and houses were displayed rapidly in random sequence. We developed a template projection approach to decode the continuous ECoG data stream spontaneously, predicting the occurrence, timing and type of visual stimulus. In this setting, we evaluated the independent and joint use of two well-studied features of brain signals, broadband changes in the frequency power spectrum of the potential and deflections in the raw potential trace (event-related potential; ERP). Our ability to predict both the timing of stimulus onset and the type of image was best when we used a combination of both the broadband response and ERP, suggesting that they capture different and complementary aspects of the subject’s perceptual state. Specifically, we were able to predict the timing and type of 96% of all stimuli, with less than 5% false positive rate and a ~20ms error in timing.     

功能性脑成像的多维模式分析方法

利用非侵入式的功能性脑成像记录大脑活动极大地提升了我们对人类认知功能的理解.与此同时,分析成像数据的手段也逐渐从传统的一元方式向更加有效的多元分析转变.在本综述中,特别针对在认知神经科学领域占主导地位的功能性磁共振成像技术,介绍其多元数据分析方法的发展以及这种分析方法的生理学基础和未来发展方向.     

Resting-State Temporal Synchronization Networks Emerge from Connectivity Topology and Heterogeneity

Spatial patterns of coherent activity across different brain areas have been identified during the resting-state fluctuations of the brain. However, recent studies indicate that resting-state activity is not stationary, but shows complex temporal dynamics. We were interested in the spatiotemporal dynamics of the phase interactions among resting-state fMRI BOLD signals from human subjects. We found that the global phase synchrony of the BOLD signals evolves on a characteristic ultra-slow (<0.01Hz) time scale, and that its temporal variations reflect the transient formation and dissolution of multiple communities of synchronized brain regions. Synchronized communities reoccurred intermittently in time and across scanning sessions. We found that the synchronization communities relate to previously defined functional networks known to be engaged in sensory-motor or cognitive function, called resting-state networks (RSNs), including the default mode network, the somato-motor network, the visual network, the auditory network, the cognitive control networks, the self-referential network, and combinations of these and other RSNs. We studied the mechanism originating the observed spatiotemporal synchronization dynamics by using a network model of phase oscillators connected through the brain’s anatomical connectivity estimated using diffusion imaging human data. The model consistently approximates the temporal and spatial synchronization patterns of the empirical data, and reveals that multiple clusters that transiently synchronize and desynchronize emerge from the complex topology of anatomical connections, provided that oscillators are heterogeneous.     

Simultaneous fMRI and EEG during the Multi-Source Interference Task

Background

fMRI and EEG are two non-invasive functional imaging techniques within cognitive neuroscience that have complementary advantages to obtain both temporal and spatial information. The multi-source interference task (MSIT) has been shown to generate robust activations of the dorsal anterior cingulate cortex (dACC) on both a single-subject level and in group averages, in fMRI studies. We have now simultaneously acquired fMRI and EEG during a cognitive interference task.

Materials and Methods

Healthy volunteers were tested in an MRI scanner with simultaneous EEG and fMRI recordings during the MSIT.

Results

The interference condition significantly increased the reaction time in the task. The fMRI analyses revealed activation of dACC as expected, in all subjects at the individual level and in group analyses. The posterior cingulate cortex was de-activated. Simultaneous EEG showed the expected anterior distribution of the interference effect, as it was restricted to frontal sites within a time frame of 80–120 ms post response.

Conclusion

The MSIT task is a reliable task for interference evaluation. fMRI shows robust activation of dACC and by adding EEG, an interference effect can be noticed within a temporal interval of 80–120 ms after the response, as a CRN (correct response negativity). This means that EEG could add a more detailed temporal aspect to the fMRI data from an interference task, and that despite the hostile environment within an MRI scanner, EEG data could be used.     

Changes in Thalamic Connectivity in the Early and Late Stages of Amnestic Mild Cognitive Impairment: A Resting-State Functional Magnetic Resonance Study from ADNI

We used resting-state functional magnetic resonance imaging (fMRI) to investigate changes in the thalamus functional connectivity in early and late stages of amnestic mild cognitive impairment. Data of 25 late stages of amnestic mild cognitive impairment (LMCI) patients, 30 early stages of amnestic mild cognitive impairment (EMCI) patients and 30 well-matched healthy controls (HC) were analyzed from the Alzheimer’s disease Neuroimaging Initiative (ADNI). We focused on the correlation between low frequency fMRI signal fluctuations in the thalamus and those in all other brain regions. Compared to healthy controls, we found functional connectivity between the left/right thalamus and a set of brain areas was decreased in LMCI and/or EMCI including right fusiform gyrus (FG), left and right superior temporal gyrus, left medial frontal gyrus extending into supplementary motor area, right insula, left middle temporal gyrus (MTG) extending into middle occipital gyrus (MOG). We also observed increased functional connectivity between the left/right thalamus and several regions in LMCI and/or EMCI including left FG, right MOG, left and right precuneus, right MTG and left inferior temporal gyrus. In the direct comparison between the LMCI and EMCI groups, we obtained several brain regions showed thalamus-seeded functional connectivity differences such as the precentral gyrus, hippocampus, FG and MTG. Briefly, these brain regions mentioned above were mainly located in the thalamo-related networks including thalamo-hippocampus, thalamo-temporal, thalamo-visual, and thalamo-default mode network. The decreased functional connectivity of the thalamus might suggest reduced functional integrity of thalamo-related networks and increased functional connectivity indicated that aMCI patients could use additional brain resources to compensate for the loss of cognitive function. Our study provided a new sight to understand the two important states of aMCI and revealed resting-state fMRI is an appropriate method for exploring pathophysiological changes in aMCI.     

Never Resting Brain: Simultaneous Representation of Two Alpha Related Processes in Humans

Brain activity is continuously modulated, even at “rest”. The alpha rhythm (8–12 Hz) has been known as the hallmark of the brain''s idle-state. However, it is still debated if the alpha rhythm reflects synchronization in a distributed network or focal generator and whether it occurs spontaneously or is driven by a stimulus. This EEG/fMRI study aimed to explore the source of alpha modulations and their distribution in the resting brain. By serendipity, while computing the individually defined power modulations of the alpha-band, two simultaneously occurring components of these modulations were found. An ‘induced alpha’ that was correlated with the paradigm (eyes open/ eyes closed), and a ‘spontaneous alpha’ that was on-going and unrelated to the paradigm. These alpha components when used as regressors for BOLD activation revealed two segregated activation maps: the ‘induced map’ included left lateral temporal cortical regions and the hippocampus; the ‘spontaneous map’ included prefrontal cortical regions and the thalamus. Our combined fMRI/EEG approach allowed to computationally untangle two parallel patterns of alpha modulations and underpin their anatomical basis in the human brain. These findings suggest that the human alpha rhythm represents at least two simultaneously occurring processes which characterize the ‘resting brain’; one is related to expected change in sensory information, while the other is endogenous and independent of stimulus change.     

Functional imaging of memory processes in humans: positron emission tomography and functional magnetic resonance imaging

Human memory is not a unitary function; it consists of multiple memory systems, with different characteristics and specialisations that are implemented in the brain. The cognitive neuroscience of human memory tries to comprehend how we encode, store, and retrieve memory items within and across those systems. The emergence of functional neuroimaging techniques offered the unprecedented opportunity to directly observe the brain regions engaged in memory functions. Brain imaging techniques can roughly be divided into those measuring the electric or magnetic fields generated by neuronal activity (EEG, magnetencephalography [MEG]) and those measuring the haemodynamic or metabolic sequelae of neuronal activity (positron emission tomography [PET], functional magnetic resonance imaging [fMRI]). Out of these techniques, the following two will be discussed in detail: fMRI and PET. Although functional neuroimaging is able to acquire images of the brain engaged in consolidating or retrieving memories, these processes are not clearly visible in the data. Statistical techniques are needed to reduce the complexity of the data and to extract the processes of interest. This article outlines the experimental and analytical procedures of neuroimaging studies with PET and fMRI. We will use a PET-study on episodic memory in human volunteers to illustrate design, analysis, and interpretation of functional imaging studies on memory.     

Transferring Cognitive Tasks Between Brain Imaging Modalities: Implications for Task Design and Results Interpretation in fMRI Studies

As cognitive neuroscience methods develop, established experimental tasks are used with emerging brain imaging modalities. Here transferring a paradigm (the visual oddball task) with a long history of behavioral and electroencephalography (EEG) experiments to a functional magnetic resonance imaging (fMRI) experiment is considered. The aims of this paper are to briefly describe fMRI and when its use is appropriate in cognitive neuroscience; illustrate how task design can influence the results of an fMRI experiment, particularly when that task is borrowed from another imaging modality; explain the practical aspects of performing an fMRI experiment. It is demonstrated that manipulating the task demands in the visual oddball task results in different patterns of blood oxygen level dependent (BOLD) activation. The nature of the fMRI BOLD measure means that many brain regions are found to be active in a particular task. Determining the functions of these areas of activation is very much dependent on task design and analysis. The complex nature of many fMRI tasks means that the details of the task and its requirements need careful consideration when interpreting data. The data show that this is particularly important in those tasks relying on a motor response as well as cognitive elements and that covert and overt responses should be considered where possible. Furthermore, the data show that transferring an EEG paradigm to an fMRI experiment needs careful consideration and it cannot be assumed that the same paradigm will work equally well across imaging modalities. It is therefore recommended that the design of an fMRI study is pilot tested behaviorally to establish the effects of interest and then pilot tested in the fMRI environment to ensure appropriate design, implementation and analysis for the effects of interest.     

Cognitive neuroscience of creativity: EEG based approaches

Cognitive neuroscience of creativity has been extensively studied using non-invasive electrical recordings from the scalp called electroencephalograms (EEGs) and event related potentials (ERPs). The paper discusses major aspects of performing research using EEG/ERP based experiments including the recording of the signals, removing noise, estimating ERP signals, and signal analysis for better understanding of the neural correlates of processes involved in creativity. Important factors to be kept in mind to record clean EEG signal in creativity research are discussed. The recorded EEG signal can be corrupted by various sources of noise and methodologies to handle the presence of unwanted artifacts and filtering noise are presented followed by methods to estimate ERPs from the EEG signals from multiple trials. The EEG and ERP signals are further analyzed using various techniques including spectral analysis, coherence analysis, and non-linear signal analysis. These analysis techniques provide a way to understand the spatial activations and temporal development of large scale electrical activity in the brain during creative tasks. The use of this methodology will further enhance our understanding the processes neural and cognitive processes involved in creativity.     

Bayesian Spatiotemporal Inference in Functional Magnetic Resonance Imaging

Mapping of the human brain by means of functional magnetic resonance imaging (fMRI) is an emerging field in cognitive and clinical neuroscience. Current techniques to detect activated areas of the brain mostly proceed in two steps. First, conventional methods of correlation, regression, and time series analysis are used to assess activation by a separate, pixelwise comparison of the fMRI signal time courses to the reference function of a presented stimulus. Spatial aspects caused by correlations between neighboring pixels are considered in a separate second step, if at all. The aim of this article is to present hierarchical Bayesian approaches that allow one to simultaneously incorporate temporal and spatial dependencies between pixels directly in the model formulation. For reasons of computational feasibility, models have to be comparatively parsimonious, without oversimplifying. We introduce parametric and semiparametric spatial and spatiotemporal models that proved appropriate and illustrate their performance applied to visual fMRI data.     

Does Older Adults’ Cognitive Function Disrupt the Malleability of Their Attitudes toward Outgroup Members?: An fMRI Investigation

In the current study we examine how individual differences in older adults’ global cognitive function impacts the extent to which their attitudes toward stigmatized individuals are malleable. Because prior research has elucidated the neural processes that are involved in evaluating stigmatized individuals who are responsible or not responsible for their condition, a cognitive neuroscience approach may be well-suited to answer this question. In the current study, 36 older and 17 young adults underwent functional magnetic resonance imaging while evaluating images of homeless people who were described as being responsible or not responsible for their condition. They also indicated how much pity they felt for each of the individuals in order to determine the extent to which their attitudes were malleable (e.g., more pity for not-responsible as compared to responsible individuals). Participants’ cognitive function and baseline measure of their attitudes toward stigmatized individuals (including homeless individuals) were assessed. Results revealed that although older adults’ attitudes were malleable, the extent to which this was true varied due to individual differences in their global cognitive function. Specifically, the difference in the magnitude of older adults’ self-reported pity for not-responsible as compared to responsible homeless individuals was predicted by their global cognitive function. Moreover, the difference in pity that older adults expressed toward not-responsible as compared to responsible homeless individuals was related to activity in the left insula and the anterior cingulate cortex (regions implicated in empathy). These results suggest that attitude malleability is affected by individual differences in global cognitive function.     

When encoding yields remembering: insights from event-related neuroimaging.

To understand human memory, it is important to determine why some experiences are remembered whereas others are forgotten. Until recently, insights into the neural bases of human memory encoding, the processes by which information is transformed into an enduring memory trace, have primarily been derived from neuropsychological studies of humans with select brain lesions. The advent of functional neuroimaging methods, such as positron emission tomography (PET) and functional magnetic resonance imaging (fMRI), has provided a new opportunity to gain additional understanding of how the brain supports memory formation. Importantly, the recent development of event-related fMRI methods now allows for examination of trial-by-trial differences in neural activity during encoding and of the consequences of these differences for later remembering. In this review, we consider the contributions of PET and fMRI studies to the understanding of memory encoding, placing a particular emphasis on recent event-related fMRI studies of the Dm effect: that is, differences in neural activity during encoding that are related to differences in subsequent memory. We then turn our attention to the rich literature on the Dm effect that has emerged from studies using event-related potentials (ERPs). It is hoped that the integration of findings from ERP studies, which offer higher temporal resolution, with those from event-related fMRI studies, which offer higher spatial resolution, will shed new light on when and why encoding yields subsequent remembering.