基于节律性脑电信号的脑-机接口

脑-机接口系统是一个不依靠外周神经和肌肉组织等而实现大脑和外界装置之间直接的交流和控制的通道。它为那些运动障碍的残疾人表达自己的意愿和实现对外部设备的控制提供了一种新的强大的技术支持。基于脑电的脑-机接口作为一种非侵入型的技术引起了该领域很多人的关注。基于脑电的脑-机接口采用了很多种类型的脑电信号。其中,振荡性的脑电图由于有较高的幅值和对噪声不敏感等特性而体现出极大的优势。也是由于这些原因,振荡性的脑电图变成了脑-机接口的应用中非常成功的设计之一。本文要介绍主要的基于脑电的脑-机接口中的两种,分别是稳态视觉诱发电位和基于运动本体感觉节律的脑-机接口。作者将详细的叙述该研究的生理背景、脑-机接口的参数,以及该系统的构造及信号处理的方法,并且会演示一些具有潜在应用价值的科研成果。     

Self-regulation of local brain activity using real-time functional magnetic resonance imaging (fMRI)

Functional magnetic resonance imaging (fMRI) measures the blood oxygen level-dependent (BOLD) signal related to neuronal activity. So far, this technique has been limited by time-consuming data analysis impeding on-line analysis. In particular, no brain-computer interface (BCI) was available which provided on-line feedback to learn physiological self-regulation of the BOLD signal. Recently, studies have shown that fMRI feedback is feasible and facilitates voluntary control of brain activity. Here we review these studies to make the fMRI feedback methodology accessible to a broader scientific community such as researchers concerned with functional brain imaging and the neurobiology of learning. Methodological and conceptual limitations were substantially reduced by artefact control, sensitivity improvements, real-time algorithms, and adapted experimental designs. Physiological self-regulation of the local BOLD response is a new paradigm for cognitive neuroscience to study brain plasticity and the functional relevance of regulated brain areas by modification of behaviour. Voluntary control of abnormal activity in circumscribed brain areas may even be applied as psychophysiological treatment.     

Artificial feedback for invasive brain–computer interfaces

During the last two decades, considerable progress has been made in the studies of brain–computer interfaces (BCIs)—devices in which motor signals from the brain are registered by multi-electrode arrays and transformed into commands for artificial actuators such as cursors and robotic devices. This review is focused on one problem. Voluntary motor control is based on neurophysiological processes, which strongly depend on the afferent innervation of skin, muscles, and joints. Thus, invasive BCI has to be based on a bidirectional system in which motor control signals are registered by multichannel microelectrodes implanted in motor areas, whereas tactile, proprioceptive, and other useful signals are transported back to the brain through spatiotemporal patterns of intracortical microstimulation (ICMS) delivered to sensory areas. In general, the studies of invasive BCIs have advanced in several directions. The progress of BCIs with artificial sensory feedback will not only help patients, but will also expand base knowledge in the field of human cortical functions.     

Comparing Different Classifiers in Sensory Motor Brain Computer Interfaces

A problem that impedes the progress in Brain-Computer Interface (BCI) research is the difficulty in reproducing the results of different papers. Comparing different algorithms at present is very difficult. Some improvements have been made by the use of standard datasets to evaluate different algorithms. However, the lack of a comparison framework still exists. In this paper, we construct a new general comparison framework to compare different algorithms on several standard datasets. All these datasets correspond to sensory motor BCIs, and are obtained from 21 subjects during their operation of synchronous BCIs and 8 subjects using self-paced BCIs. Other researchers can use our framework to compare their own algorithms on their own datasets. We have compared the performance of different popular classification algorithms over these 29 subjects and performed statistical tests to validate our results. Our findings suggest that, for a given subject, the choice of the classifier for a BCI system depends on the feature extraction method used in that BCI system. This is in contrary to most of publications in the field that have used Linear Discriminant Analysis (LDA) as the classifier of choice for BCI systems.     

Resting-State fMRI Activity Predicts Unsupervised Learning and Memory in an Immersive Virtual Reality Environment

In the real world, learning often proceeds in an unsupervised manner without explicit instructions or feedback. In this study, we employed an experimental paradigm in which subjects explored an immersive virtual reality environment on each of two days. On day 1, subjects implicitly learned the location of 39 objects in an unsupervised fashion. On day 2, the locations of some of the objects were changed, and object location recall performance was assessed and found to vary across subjects. As prior work had shown that functional magnetic resonance imaging (fMRI) measures of resting-state brain activity can predict various measures of brain performance across individuals, we examined whether resting-state fMRI measures could be used to predict object location recall performance. We found a significant correlation between performance and the variability of the resting-state fMRI signal in the basal ganglia, hippocampus, amygdala, thalamus, insula, and regions in the frontal and temporal lobes, regions important for spatial exploration, learning, memory, and decision making. In addition, performance was significantly correlated with resting-state fMRI connectivity between the left caudate and the right fusiform gyrus, lateral occipital complex, and superior temporal gyrus. Given the basal ganglia''s role in exploration, these findings suggest that tighter integration of the brain systems responsible for exploration and visuospatial processing may be critical for learning in a complex environment.     

Evolution of brain-computer interface: action potentials, local field potentials and electrocorticograms

Brain computer interfaces (BCIs) were originally developed to give severely motor impaired patients a method to communicate and interact with their environment. Initially most BCI systems were based on non-invasive electroencephalographic recordings from the surface of the scalp. To increase control speed, accuracy and complexity, researchers began utilizing invasive recording modalities. BCIs using multi-single unit action potentials have provided elegant multi-dimensional control of both computer cursors and robotic limbs in the last few years. However, long-term stability issues with single-unit arrays has lead researchers to investigate other invasive recording modalities such as high-frequency local field potentials and electrocorticography (ECoG). Although ECoG originally evolved as a replacement for single-unit BCIs, it has come full circle to become an effective tool for studying cortical neurophysiology.     

A reductionist approach to the analysis of learning in brain–computer interfaces

The complexity and scale of brain–computer interface (BCI) studies limit our ability to investigate how humans learn to use BCI systems. It also limits our capacity to develop adaptive algorithms needed to assist users with their control. Adaptive algorithm development is forced offline and typically uses static data sets. But this is a poor substitute for the online, dynamic environment where algorithms are ultimately deployed and interact with an adapting user. This work evaluates a paradigm that simulates the control problem faced by human subjects when controlling a BCI, but which avoids the many complications associated with full-scale BCI studies. Biological learners can be studied in a reductionist way as they solve BCI-like control problems, and machine learning algorithms can be developed and tested in closed loop with the subjects before being translated to full BCIs. The method is to map 19 joint angles of the hand (representing neural signals) to the position of a 2D cursor which must be piloted to displayed targets (a typical BCI task). An investigation is presented on how closely the joint angle method emulates BCI systems; a novel learning algorithm is evaluated, and a performance difference between genders is discussed.     

Voluntary Enhancement of Neural Signatures of Affiliative Emotion Using fMRI Neurofeedback

In Ridley Scott’s film “Blade Runner”, empathy-detection devices are employed to measure affiliative emotions. Despite recent neurocomputational advances, it is unknown whether brain signatures of affiliative emotions, such as tenderness/affection, can be decoded and voluntarily modulated. Here, we employed multivariate voxel pattern analysis and real-time fMRI to address this question. We found that participants were able to use visual feedback based on decoded fMRI patterns as a neurofeedback signal to increase brain activation characteristic of tenderness/affection relative to pride, an equally complex control emotion. Such improvement was not observed in a control group performing the same fMRI task without neurofeedback. Furthermore, the neurofeedback-driven enhancement of tenderness/affection-related distributed patterns was associated with local fMRI responses in the septohypothalamic area and frontopolar cortex, regions previously implicated in affiliative emotion. This demonstrates that humans can voluntarily enhance brain signatures of tenderness/affection, unlocking new possibilities for promoting prosocial emotions and countering antisocial behavior.     

A real-time fMRI-based spelling device immediately enabling robust motor-independent communication

Human communication entirely depends on the functional integrity of the neuromuscular system. This is devastatingly illustrated in clinical conditions such as the so-called locked-in syndrome (LIS), in which severely motor-disabled patients become incapable to communicate naturally--while being fully conscious and awake. For the last 20 years, research on motor-independent communication has focused on developing brain-computer interfaces (BCIs) implementing neuroelectric signals for communication (e.g., [2-7]), and BCIs based on electroencephalography (EEG) have already been applied successfully to concerned patients. However, not all patients achieve proficiency in EEG-based BCI control. Thus, more recently, hemodynamic brain signals have also been explored for BCI purposes. Here, we introduce the first spelling device based on fMRI. By exploiting spatiotemporal characteristics of hemodynamic responses, evoked by performing differently timed mental imagery tasks, our novel letter encoding technique allows translating any freely chosen answer (letter-by-letter) into reliable and differentiable single-trial fMRI signals. Most importantly, automated letter decoding in real time enables back-and-forth communication within a single scanning session. Because the suggested spelling device requires only little effort and pretraining, it is immediately operational and possesses high potential for clinical applications, both in terms of diagnostics and establishing short-term communication with nonresponsive and severely motor-impaired patients.     

Motor Imagery for Severely Motor-Impaired Patients: Evidence for Brain-Computer Interfacing as Superior Control Solution

Brain-Computer Interfaces (BCIs) strive to decode brain signals into control commands for severely handicapped people with no means of muscular control. These potential users of noninvasive BCIs display a large range of physical and mental conditions. Prior studies have shown the general applicability of BCI with patients, with the conflict of either using many training sessions or studying only moderately restricted patients. We present a BCI system designed to establish external control for severely motor-impaired patients within a very short time. Within only six experimental sessions, three out of four patients were able to gain significant control over the BCI, which was based on motor imagery or attempted execution. For the most affected patient, we found evidence that the BCI could outperform the best assistive technology (AT) of the patient in terms of control accuracy, reaction time and information transfer rate. We credit this success to the applied user-centered design approach and to a highly flexible technical setup. State-of-the art machine learning methods allowed the exploitation and combination of multiple relevant features contained in the EEG, which rapidly enabled the patients to gain substantial BCI control. Thus, we could show the feasibility of a flexible and tailorable BCI application in severely disabled users. This can be considered a significant success for two reasons: Firstly, the results were obtained within a short period of time, matching the tight clinical requirements. Secondly, the participating patients showed, compared to most other studies, very severe communication deficits. They were dependent on everyday use of AT and two patients were in a locked-in state. For the most affected patient a reliable communication was rarely possible with existing AT.     

Review of brain encoding and decoding mechanisms for EEG-based brain–computer interface

A brain–computer interface (BCI) can connect humans and machines directly and has achieved successful applications in the past few decades. Many new BCI paradigms and algorithms have been developed in recent years. Therefore, it is necessary to review new progress in BCIs. This paper summarizes progress for EEG-based BCIs from the perspective of encoding paradigms and decoding algorithms, which are two key elements of BCI systems. Encoding paradigms are grouped by their underlying neural meachanisms, namely sensory- and motor-related, vision-related, cognition-related and hybrid paradigms. Decoding algorithms are reviewed in four categories, namely decomposition algorithms, Riemannian geometry, deep learning and transfer learning. This review will provide a comprehensive overview of both modern primary paradigms and algorithms, making it helpful for those who are developing BCI systems.     

Robust Brain-computer Interface for virtual Keyboard (RoBIK): Project results

Brain-Computer Interface (BCI) is a technology that translates the brain electrical activity into a command for a device such as a robotic arm, a wheelchair or a spelling device. BCIs have long been described as an assistive technology for severely disabled patients because they completely bypass the need for muscular activity. The clinical reality is however dramatically different and most patients who use BCIs today are doing so as part of constraining clinical trials. To achieve the technological transfer from bench to bedside, BCI must gain ease of use and robustness of both measure (electroencephalography [EEG]) and interface (signal processing and applications). The Robust Brain-computer Interface for virtual Keyboard (RoBIK) project aimed at the development of a BCI system for communication that could be used on a daily basis by patients without the help of a trained team of researchers. To guide further developments clinicians first assessed patients’ needs. The prototype subsequently developed consisted in a 14 felt-pad electrodes EEG headset sampling at 256 Hz by an electronic component capable of transmitting signals wirelessly. The application was a virtual keyboard generating a novel stimulation paradigm to elicit P300 Evoked Related Potentials (ERPs) for communication. Raw EEG signals were treated with OpenViBE open-source software including novel signal processing and stimulation techniques.     

Large-Scale Assessment of a Fully Automatic Co-Adaptive Motor Imagery-Based Brain Computer Interface

In the last years Brain Computer Interface (BCI) technology has benefited from the development of sophisticated machine leaning methods that let the user operate the BCI after a few trials of calibration. One remarkable example is the recent development of co-adaptive techniques that proved to extend the use of BCIs also to people not able to achieve successful control with the standard BCI procedure. Especially for BCIs based on the modulation of the Sensorimotor Rhythm (SMR) these improvements are essential, since a not negligible percentage of users is unable to operate SMR-BCIs efficiently. In this study we evaluated for the first time a fully automatic co-adaptive BCI system on a large scale. A pool of 168 participants naive to BCIs operated the co-adaptive SMR-BCI in one single session. Different psychological interventions were performed prior the BCI session in order to investigate how motor coordination training and relaxation could influence BCI performance. A neurophysiological indicator based on the Power Spectral Density (PSD) was extracted by the recording of few minutes of resting state brain activity and tested as predictor of BCI performances. Results show that high accuracies in operating the BCI could be reached by the majority of the participants before the end of the session. BCI performances could be significantly predicted by the neurophysiological indicator, consolidating the validity of the model previously developed. Anyway, we still found about 22% of users with performance significantly lower than the threshold of efficient BCI control at the end of the session. Being the inter-subject variability still the major problem of BCI technology, we pointed out crucial issues for those who did not achieve sufficient control. Finally, we propose valid developments to move a step forward to the applicability of the promising co-adaptive methods.     

Influence of Stimuli Spatial Proximity on a SSVEP-Based BCI Performance

Objective: Steady-State Visual Evoked Potentials based Brain-Computer Interfaces (SSVEP-based BCIs) systems have been shown as promising technology due to their short response time and ease of use. SSVEP-based BCIs use brain responses to a flickering visual stimulus as an input command to an external application or device, and it can be influenced by stimulus properties, signal recording, and signal processing. We aim to investigate the system performance varying the stimuli spatial proximity (a stimulus property).Material and methods: We performed a comparative analysis of two visual interface designs (named cross and square) for an SSVEP-based BCI. The power spectrum density (PSD) was used as feature extraction and the Support Machine Vector (SVM) as classification method. We also analyzed the effects of five flickering frequencies (6.67, 8.57, 10, 12 e 15 Hz) between and within interfaces.Results: We found higher accuracy rates for the flickering frequencies of 10, 12, and 15 Hz. The stimulus of 10 Hz presented the highest SSVEP amplitude response for both interfaces. The system presented the best performance (highest classification accuracy and information transfer rate) using the cross interface (lower visual angle).Conclusion: Our findings suggest that the system has the highest performance in the spatial proximity range from 4° to 13° (visual angle). In addition, we conclude that as the stimulus spatial proximity increases, the interference from other stimuli reduces, and the SSVEP amplitude response decreases, which reduces system accuracy. The inter-stimulus distance is a visual interface parameter that must be chosen carefully to increase the efficiency of an SSVEP-based BCI.     

Estimating the Intended Sound Direction of the User: Toward an Auditory Brain-Computer Interface Using Out-of-Head Sound Localization

The auditory Brain-Computer Interface (BCI) using electroencephalograms (EEG) is a subject of intensive study. As a cue, auditory BCIs can deal with many of the characteristics of stimuli such as tone, pitch, and voices. Spatial information on auditory stimuli also provides useful information for a BCI. However, in a portable system, virtual auditory stimuli have to be presented spatially through earphones or headphones, instead of loudspeakers. We investigated the possibility of an auditory BCI using the out-of-head sound localization technique, which enables us to present virtual auditory stimuli to users from any direction, through earphones. The feasibility of a BCI using this technique was evaluated in an EEG oddball experiment and offline analysis. A virtual auditory stimulus was presented to the subject from one of six directions. Using a support vector machine, we were able to classify whether the subject attended the direction of a presented stimulus from EEG signals. The mean accuracy across subjects was 70.0% in the single-trial classification. When we used trial-averaged EEG signals as inputs to the classifier, the mean accuracy across seven subjects reached 89.5% (for 10-trial averaging). Further analysis showed that the P300 event-related potential responses from 200 to 500 ms in central and posterior regions of the brain contributed to the classification. In comparison with the results obtained from a loudspeaker experiment, we confirmed that stimulus presentation by out-of-head sound localization achieved similar event-related potential responses and classification performances. These results suggest that out-of-head sound localization enables us to provide a high-performance and loudspeaker-less portable BCI system.     

Measuring Granger Causality between cortical regions from voxelwise fMRI BOLD signals with LASSO

Functional brain network studies using the Blood Oxygen-Level Dependent (BOLD) signal from functional Magnetic Resonance Imaging (fMRI) are becoming increasingly prevalent in research on the neural basis of human cognition. An important problem in functional brain network analysis is to understand directed functional interactions between brain regions during cognitive performance. This problem has important implications for understanding top-down influences from frontal and parietal control regions to visual occipital cortex in visuospatial attention, the goal motivating the present study. A common approach to measuring directed functional interactions between two brain regions is to first create nodal signals by averaging the BOLD signals of all the voxels in each region, and to then measure directed functional interactions between the nodal signals. Another approach, that avoids averaging, is to measure directed functional interactions between all pairwise combinations of voxels in the two regions. Here we employ an alternative approach that avoids the drawbacks of both averaging and pairwise voxel measures. In this approach, we first use the Least Absolute Shrinkage Selection Operator (LASSO) to pre-select voxels for analysis, then compute a Multivariate Vector AutoRegressive (MVAR) model from the time series of the selected voxels, and finally compute summary Granger Causality (GC) statistics from the model to represent directed interregional interactions. We demonstrate the effectiveness of this approach on both simulated and empirical fMRI data. We also show that averaging regional BOLD activity to create a nodal signal may lead to biased GC estimation of directed interregional interactions. The approach presented here makes it feasible to compute GC between brain regions without the need for averaging. Our results suggest that in the analysis of functional brain networks, careful consideration must be given to the way that network nodes and edges are defined because those definitions may have important implications for the validity of the analysis.     

The P300 based brain-computer interface: effect of stimulus position in a stimulus train

The P300 brain-computer interface (BCI) is currently the most efficient BCI. This interface is based on detection of the P300 wave of the brain potentials evoked when a symbol related to the intended input is highlighted. To increase operation speed of the P300 BCI, reduction of the number of stimuli repetitions is needed. This reduction leads to increase of the relative contribution to the input symbol detection from the reaction to the first target stimulus. It is known that the event-related potentials (ERP) to the first stimulus presentations can be different from the ERP to stimuli presented latter. In particular, the amplitude of responses to the first stimulus presentations is often increased, which is beneficial for their recognition by the BCI. However, this effect was not studied within the BCI framework. The current study examined the ERP obtained from healthy participants (n = 14) in the standard P300 BCI paradigm using 10 trials, as well as in the modified P300 BCI with stimuli presented on moving objects in triple-trial (n = 6) and single-trial (n = 6) stimulation modes. Increased ERP amplitude was observed in response to the first target stimuli in both conditions, as well as in the single-trial mode comparing to triple-trial. We discuss the prospects of using the specific features of the ERP to first stimuli and the single-trial ERP for optimizing the high-speed modes in the P300 BCIs.     

FMRI Effective Connectivity and TMS Chronometry: Complementary Accounts of Causality in the Visuospatial Judgment Network

Background

While traditionally quite distinct, functional neuroimaging (e.g. functional magnetic resonance imaging: fMRI) and functional interference techniques (e.g. transcranial magnetic stimulation: TMS) increasingly address similar questions of functional brain organization, including connectivity, interactions, and causality in the brain. Time-resolved TMS over multiple brain network nodes can elucidate the relative timings of functional relevance for behavior (“TMS chronometry”), while fMRI functional or effective connectivity (fMRI EC) can map task-specific interactions between brain regions based on the interrelation of measured signals. The current study empirically assessed the relation between these different methods.

Methodology/Principal Findings

One group of 15 participants took part in two experiments: one fMRI EC study, and one TMS chronometry study, both of which used an established cognitive paradigm involving one visuospatial judgment task and one color judgment control task. Granger causality mapping (GCM), a data-driven variant of fMRI EC analysis, revealed a frontal-to-parietal flow of information, from inferior/middle frontal gyrus (MFG) to posterior parietal cortex (PPC). FMRI EC-guided Neuronavigated TMS had behavioral effects when applied to both PPC and to MFG, but the temporal pattern of these effects was similar for both stimulation sites. At first glance, this would seem in contradiction to the fMRI EC results. However, we discuss how TMS chronometry and fMRI EC are conceptually different and show how they can be complementary and mutually constraining, rather than contradictory, on the basis of our data.

Conclusions/Significance

The findings that fMRI EC could successfully localize functionally relevant TMS target regions on the single subject level, and conversely, that TMS confirmed an fMRI EC identified functional network to be behaviorally relevant, have important methodological and theoretical implications. Our results, in combination with data from earlier studies by our group (Sack et al., 2007, Cerebral Cortex), lead to informed speculations on complex brain mechanisms, and TMS disruption thereof, underlying visuospatial judgment. This first in-depth empirical and conceptual comparison of fMRI EC and TMS chronometry thereby shows the complementary insights offered by the two methods.     

Dynamic Adjustment of Stimuli in Real Time Functional Magnetic Resonance Imaging

The conventional fMRI image analysis approach to associating stimuli to brain activation is performed by carrying out a massive number of parallel univariate regression analyses. fMRI blood-oxygen-level dependent (BOLD) signal, the basis of these analyses, is known for its low signal-noise-ratio and high spatial and temporal signal correlation. In order to ensure accurate localization of brain activity, stimulus administration in an fMRI session is often lengthy and repetitive. Real-time fMRI BOLD signal analysis is carried out as the signal is observed. This method allows for dynamic, real-time adjustment of stimuli through sequential experimental designs. We have developed a voxel-wise sequential probability ratio test (SPRT) approach for dynamically determining localization, as well as decision rules for stopping stimulus administration. SPRT methods and general linear model (GLM) approaches are combined to identify brain regions that are activated by specific elements of stimuli. Stimulus administration is dynamically stopped when sufficient statistical evidence is collected to determine activation status across regions of interest, following predetermined statistical error thresholds. Simulation experiments and an example based on real fMRI data show that scan volumes can be substantially reduced when compared with pre-determined, fixed designs while achieving similar or better accuracy in detecting activated voxels. Moreover, the proposed approach is also able to accurately detect differentially activated areas, and other comparisons between task-related GLM parameters that can be formulated in a hypothesis-testing framework. Finally, we give a demonstration of SPRT being employed in conjunction with a halving algorithm to dynamically adjust stimuli.     

Voxel-Wise Motion Artifacts in Population-Level Whole-Brain Connectivity Analysis of Resting-State fMRI

Functional Magnetic Resonance Imaging (fMRI) based brain connectivity analysis maps the functional networks of the brain by estimating the degree of synchronous neuronal activity between brain regions. Recent studies have demonstrated that “resting-state” fMRI-based brain connectivity conclusions may be erroneous when motion artifacts have a differential effect on fMRI BOLD signals for between group comparisons. A potential explanation could be that in-scanner displacement, due to rotational components, is not spatially constant in the whole brain. However, this localized nature of motion artifacts is poorly understood and is rarely considered in brain connectivity studies. In this study, we initially demonstrate the local correspondence between head displacement and the changes in the resting-state fMRI BOLD signal. Than, we investigate how connectivity strength is affected by the population-level variation in the spatial pattern of regional displacement. We introduce Regional Displacement Interaction (RDI), a new covariate parameter set for second-level connectivity analysis and demonstrate its effectiveness in reducing motion related confounds in comparisons of groups with different voxel-vise displacement pattern and preprocessed using various nuisance regression methods. The effect of using RDI as second-level covariate is than demonstrated in autism-related group comparisons. The relationship between the proposed method and some of the prevailing subject-level nuisance regression techniques is evaluated. Our results show that, depending on experimental design, treating in-scanner head motion as a global confound may not be appropriate. The degree of displacement is highly variable among various brain regions, both within and between subjects. These regional differences bias correlation-based measures of brain connectivity. The inclusion of the proposed second-level covariate into the analysis successfully reduces artifactual motion-related group differences and preserves real neuronal differences, as demonstrated by the autism-related comparisons.