A collaborative brain-computer interface for improving human performance

Electroencephalogram (EEG) based brain-computer interfaces (BCI) have been studied since the 1970s. Currently, the main focus of BCI research lies on the clinical use, which aims to provide a new communication channel to patients with motor disabilities to improve their quality of life. However, the BCI technology can also be used to improve human performance for normal healthy users. Although this application has been proposed for a long time, little progress has been made in real-world practices due to technical limits of EEG. To overcome the bottleneck of low single-user BCI performance, this study proposes a collaborative paradigm to improve overall BCI performance by integrating information from multiple users. To test the feasibility of a collaborative BCI, this study quantitatively compares the classification accuracies of collaborative and single-user BCI applied to the EEG data collected from 20 subjects in a movement-planning experiment. This study also explores three different methods for fusing and analyzing EEG data from multiple subjects: (1) Event-related potentials (ERP) averaging, (2) Feature concatenating, and (3) Voting. In a demonstration system using the Voting method, the classification accuracy of predicting movement directions (reaching left vs. reaching right) was enhanced substantially from 66% to 80%, 88%, 93%, and 95% as the numbers of subjects increased from 1 to 5, 10, 15, and 20, respectively. Furthermore, the decision of reaching direction could be made around 100-250 ms earlier than the subject's actual motor response by decoding the ERP activities arising mainly from the posterior parietal cortex (PPC), which are related to the processing of visuomotor transmission. Taken together, these results suggest that a collaborative BCI can effectively fuse brain activities of a group of people to improve the overall performance of natural human behavior.     

Evaluating True BCI Communication Rate through Mutual Information and Language Models

Brain-computer interface (BCI) systems are a promising means for restoring communication to patients suffering from “locked-in” syndrome. Research to improve system performance primarily focuses on means to overcome the low signal to noise ratio of electroencephalogric (EEG) recordings. However, the literature and methods are difficult to compare due to the array of evaluation metrics and assumptions underlying them, including that: 1) all characters are equally probable, 2) character selection is memoryless, and 3) errors occur completely at random. The standardization of evaluation metrics that more accurately reflect the amount of information contained in BCI language output is critical to make progress. We present a mutual information-based metric that incorporates prior information and a model of systematic errors. The parameters of a system used in one study were re-optimized, showing that the metric used in optimization significantly affects the parameter values chosen and the resulting system performance. The results of 11 BCI communication studies were then evaluated using different metrics, including those previously used in BCI literature and the newly advocated metric. Six studies'' results varied based on the metric used for evaluation and the proposed metric produced results that differed from those originally published in two of the studies. Standardizing metrics to accurately reflect the rate of information transmission is critical to properly evaluate and compare BCI communication systems and advance the field in an unbiased manner.     

Brain-Computer Interface Speller System for Alternative Communication: A Review

Brain-computer interface (BCI) speller is a system that provides an alternative communication for the disable people. The brain wave is translated into machine command through a BCI speller which can be used as a communication medium for the patients to express their thought without any motor movement. A BCI speller aims to spell characters by using the electroencephalogram (EEG) signal. Several types of BCI spellers are available based on the EEG signal. A standard BCI speller system consists of the following elements: BCI speller paradigm, data acquisition system and signal processing algorithms. In this work, a systematic review is provided on the BCI speller system and it includes speller paradigms, feature extraction, feature optimization and classification techniques for BCI speller. The advantages and limitations of different speller paradigm and machine learning algorithms are discussed in this article. Also, the future research directions are discussed which can overcome the limitations of present state-of-the-art techniques for BCI speller.     

Model analyses of visual biofeedback training for EEG-based brain-computer interface

The primary goal of this study was to construct a simulation model of a biofeedback brain-computer interface (BCI) system to analyze the effect of biofeedback training on BCI users. A mathematical model of a man-machine visual-biofeedback BCI system was constructed to simulate a subject using a BCI system to control cursor movements. The model consisted of a visual tracking system, a thalamo-cortical model for EEG generation, and a BCI system. The BCI system in the model was realized for real experiments of visual biofeedback training. Ten sessions of visual biofeedback training were performed in eight normal subjects during a 3-week period. The task was to move a cursor horizontally across a screen, or to hold it at the screen’s center. Experimental conditions and EEG data obtained from real experiments were then simulated with the model. Three model parameters, representing the adaptation rate of gain in the visual tracking system and the relative synaptic strength between the thalamic reticular and thalamo-cortical cells in the Rolandic areas, were estimated by optimization techniques so that the performance of the model best fitted the experimental results. The serial changes of these parameters over the ten sessions, reflecting the effects of biofeedback training, were analyzed. The model simulation could reproduce results similar to the experimental data. The group mean success rate and information transfer rate improved significantly after training (56.6 to 81.1% and 0.19 to 0.76 bits/trial, respectively). All three model parameters displayed similar and statistically significant increasing trends with time. Extensive simulation with systematic changes of these parameters also demonstrated that assigning larger values to the parameters improved the BCI performance. We constructed a model of a biofeedback BCI system that could simulate experimental data and the effect of training. The simulation results implied that the improvement was achieved through a quicker adaptation rate in visual tracking gain and a larger synaptic gain from the visual tracking system to the thalamic reticular cells. In addition to the purpose of this study, the constructed biofeedback BCI model can also be used both to investigate the effects of different biofeedback paradigms and to test, estimate, or predict the performances of other newly developed BCI signal processing algorithms.     

基于多特征融合的多分类运动想象脑电信号识别研究

针对目前多分类运动想象脑电识别存在特征提取单一、分类准确率低等问题,提出一种多特征融合的四分类运动想象脑电识别方法来提高识别率。对预处理后的脑电信号分别使用希尔伯特-黄变换、一对多共空间模式、近似熵、模糊熵、样本熵提取结合时频—空域—非线性动力学的初始特征向量,用主成分分析降维,最后使用粒子群优化支持向量机分类。该算法通过对国际标准数据集BCI2005 Data set IIIa中的k3b受试者数据经MATLAB仿真处理后获得93.30%的识别率,均高于单一特征和其它组合特征下的识别率。分别对四名实验者实验采集运动想象脑电数据,使用本研究提出的方法处理获得了72.96%的平均识别率。结果表明多特征融合的特征提取方法能更好的表征运动想象脑电信号,使用粒子群支持向量机可取得较高的识别准确率,为人脑的认知活动提供了一种新的识别方法。     

Estimating the Intended Sound Direction of the User: Toward an Auditory Brain-Computer Interface Using Out-of-Head Sound Localization

The auditory Brain-Computer Interface (BCI) using electroencephalograms (EEG) is a subject of intensive study. As a cue, auditory BCIs can deal with many of the characteristics of stimuli such as tone, pitch, and voices. Spatial information on auditory stimuli also provides useful information for a BCI. However, in a portable system, virtual auditory stimuli have to be presented spatially through earphones or headphones, instead of loudspeakers. We investigated the possibility of an auditory BCI using the out-of-head sound localization technique, which enables us to present virtual auditory stimuli to users from any direction, through earphones. The feasibility of a BCI using this technique was evaluated in an EEG oddball experiment and offline analysis. A virtual auditory stimulus was presented to the subject from one of six directions. Using a support vector machine, we were able to classify whether the subject attended the direction of a presented stimulus from EEG signals. The mean accuracy across subjects was 70.0% in the single-trial classification. When we used trial-averaged EEG signals as inputs to the classifier, the mean accuracy across seven subjects reached 89.5% (for 10-trial averaging). Further analysis showed that the P300 event-related potential responses from 200 to 500 ms in central and posterior regions of the brain contributed to the classification. In comparison with the results obtained from a loudspeaker experiment, we confirmed that stimulus presentation by out-of-head sound localization achieved similar event-related potential responses and classification performances. These results suggest that out-of-head sound localization enables us to provide a high-performance and loudspeaker-less portable BCI system.     

Robust Brain-computer Interface for virtual Keyboard (RoBIK): Project results

Brain-Computer Interface (BCI) is a technology that translates the brain electrical activity into a command for a device such as a robotic arm, a wheelchair or a spelling device. BCIs have long been described as an assistive technology for severely disabled patients because they completely bypass the need for muscular activity. The clinical reality is however dramatically different and most patients who use BCIs today are doing so as part of constraining clinical trials. To achieve the technological transfer from bench to bedside, BCI must gain ease of use and robustness of both measure (electroencephalography [EEG]) and interface (signal processing and applications). The Robust Brain-computer Interface for virtual Keyboard (RoBIK) project aimed at the development of a BCI system for communication that could be used on a daily basis by patients without the help of a trained team of researchers. To guide further developments clinicians first assessed patients’ needs. The prototype subsequently developed consisted in a 14 felt-pad electrodes EEG headset sampling at 256 Hz by an electronic component capable of transmitting signals wirelessly. The application was a virtual keyboard generating a novel stimulation paradigm to elicit P300 Evoked Related Potentials (ERPs) for communication. Raw EEG signals were treated with OpenViBE open-source software including novel signal processing and stimulation techniques.     

LASSO based stimulus frequency recognition model for SSVEP BCIs

Steady-state visual evoked potential (SSVEP) has been increasingly used for the study of brain–computer interface (BCI). How to recognize SSVEP with shorter time and lower error rate is one of the key points to develop a more efficient SSVEP-based BCI. To achieve this goal, we make use of the sparsity constraint of the least absolute shrinkage and selection operator (LASSO) for the extraction of more discriminative features of SSVEP, and then we propose a LASSO model using the linear regression between electroencephalogram (EEG) recordings and the standard square-wave signals of different frequencies to recognize SSVEP without the training stage. In this study, we verified the proposed LASSO model offline with the EEG data of nine healthy subjects in contrast to canonical correlation analysis (CCA). In the experiment, when a shorter time window was used, we found that the LASSO model yielded better performance in extracting robust and detectable features of SSVEP, and the information transfer rate obtained by the LASSO model was significantly higher than that of the CCA. Our proposed method can assist to reduce the recording time without sacrificing the classification accuracy and is promising for a high-speed SSVEP-based BCI.     

EEG power spectral density in locked-in and completely locked-in state patients: a longitudinal study

Persons with their eye closed and without any means of communication is said to be in a completely locked-in state (CLIS) while when they could still open their eyes actively or passively and have some means of communication are said to be in locked-in state (LIS). Two patients in CLIS without any means of communication, and one patient in the transition from LIS to CLIS with means of communication, who have Amyotrophic Lateral Sclerosis were followed at a regular interval for more than 1 year. During each visit, resting-state EEG was recorded before the brain–computer interface (BCI) based communication sessions. The resting-state EEG of the patients was analyzed to elucidate the evolution of their EEG spectrum over time with the disease’s progression to provide future BCI-research with the relevant information to classify changes in EEG evolution. Comparison of power spectral density (PSD) of these patients revealed a significant difference in the PSD’s of patients in CLIS without any means of communication and the patient in the transition from LIS to CLIS with means of communication. The EEG of patients without any means of communication is devoid of alpha, beta, and higher frequencies than the patient in transition who still had means of communication. The results show that the change in the EEG frequency spectrum may serve as an indicator of the communication ability of such patients.Electronic supplementary materialThe online version of this article (10.1007/s11571-020-09639-w) contains supplementary material, which is available to authorized users.     

Hidden Markov models used for the offline classification of EEG data.

Hidden Markov models (HMM) are introduced for the offline classification of single-trail EEG data in a brain-computer-interface (BCI). The HMMs are used to classify Hjorth parameters calculated from bipolar EEG data, recorded during the imagination of a left or right hand movement. The effects of different types of HMMs on the recognition rate are discussed. Furthermore a comparison of the results achieved with the linear discriminant (LD) and the HMM, is presented.     

Motor Imagery for Severely Motor-Impaired Patients: Evidence for Brain-Computer Interfacing as Superior Control Solution

Brain-Computer Interfaces (BCIs) strive to decode brain signals into control commands for severely handicapped people with no means of muscular control. These potential users of noninvasive BCIs display a large range of physical and mental conditions. Prior studies have shown the general applicability of BCI with patients, with the conflict of either using many training sessions or studying only moderately restricted patients. We present a BCI system designed to establish external control for severely motor-impaired patients within a very short time. Within only six experimental sessions, three out of four patients were able to gain significant control over the BCI, which was based on motor imagery or attempted execution. For the most affected patient, we found evidence that the BCI could outperform the best assistive technology (AT) of the patient in terms of control accuracy, reaction time and information transfer rate. We credit this success to the applied user-centered design approach and to a highly flexible technical setup. State-of-the art machine learning methods allowed the exploitation and combination of multiple relevant features contained in the EEG, which rapidly enabled the patients to gain substantial BCI control. Thus, we could show the feasibility of a flexible and tailorable BCI application in severely disabled users. This can be considered a significant success for two reasons: Firstly, the results were obtained within a short period of time, matching the tight clinical requirements. Secondly, the participating patients showed, compared to most other studies, very severe communication deficits. They were dependent on everyday use of AT and two patients were in a locked-in state. For the most affected patient a reliable communication was rarely possible with existing AT.     

A Comparison Study of Canonical Correlation Analysis Based Methods for Detecting Steady-State Visual Evoked Potentials

Canonical correlation analysis (CCA) has been widely used in the detection of the steady-state visual evoked potentials (SSVEPs) in brain-computer interfaces (BCIs). The standard CCA method, which uses sinusoidal signals as reference signals, was first proposed for SSVEP detection without calibration. However, the detection performance can be deteriorated by the interference from the spontaneous EEG activities. Recently, various extended methods have been developed to incorporate individual EEG calibration data in CCA to improve the detection performance. Although advantages of the extended CCA methods have been demonstrated in separate studies, a comprehensive comparison between these methods is still missing. This study performed a comparison of the existing CCA-based SSVEP detection methods using a 12-class SSVEP dataset recorded from 10 subjects in a simulated online BCI experiment. Classification accuracy and information transfer rate (ITR) were used for performance evaluation. The results suggest that individual calibration data can significantly improve the detection performance. Furthermore, the results showed that the combination method based on the standard CCA and the individual template based CCA (IT-CCA) achieved the highest performance.     

Analyzing text recognition from tactually evoked EEG

Tactual exploration of objects produce specific patterns in the human brain and hence objects can be recognized by analyzing brain signals during tactile exploration. The present work aims at analyzing EEG signals online for recognition of embossed texts by tactual exploration. EEG signals are acquired from the parietal region over the somatosensory cortex of blindfolded healthy subjects while they tactually explored embossed texts, including symbols, numbers, and alphabets. Classifiers based on the principle of supervised learning are trained on the extracted EEG feature space, comprising three features, namely, adaptive autoregressive parameters, Hurst exponents, and power spectral density, to recognize the respective texts. The pre-trained classifiers are used to classify the EEG data to identify the texts online and the recognized text is displayed on the computer screen for communication. Online classifications of two, four, and six classes of embossed texts are achieved with overall average recognition rates of 76.62, 72.31, and 67.62% respectively and the computational time is less than 2 s in each case. The maximum information transfer rate and utility of the system performance over all experiments are 0.7187 and 2.0529 bits/s respectively. This work presents a study that shows the possibility to classify 3D letters using tactually evoked EEG. In future, it will help the BCI community to design stimuli for better tactile augmentation n also opens new directions of research to facilitate 3D letters for visually impaired persons. Further, 3D maps can be generated for aiding tactual BCI in teleoperation.     

Design of an EEG-based brain-computer interface (BCI) from standard components running in real-time under Windows.

An EEG-based brain-computer interface (BCI) is a direct connection between the human brain and the computer. Such a communication system is needed by patients with severe motor impairments (e.g. late stage of Amyotrophic Lateral Sclerosis) and has to operate in real-time. This paper describes the selection of the appropriate components to construct such a BCI and focuses also on the selection of a suitable programming language and operating system. The multichannel system runs under Windows 95, equipped with a real-time Kernel expansion to obtain reasonable real-time operations on a standard PC. Matlab controls the data acquisition and the presentation of the experimental paradigm, while Simulink is used to calculate the recursive least square (RLS) algorithm that describes the current state of the EEG in real-time. First results of the new low-cost BCI show that the accuracy of differentiating imagination of left and right hand movement is around 95%.     

基于节律性脑电信号的脑-机接口

脑-机接口系统是一个不依靠外周神经和肌肉组织等而实现大脑和外界装置之间直接的交流和控制的通道。它为那些运动障碍的残疾人表达自己的意愿和实现对外部设备的控制提供了一种新的强大的技术支持。基于脑电的脑-机接口作为一种非侵入型的技术引起了该领域很多人的关注。基于脑电的脑-机接口采用了很多种类型的脑电信号。其中,振荡性的脑电图由于有较高的幅值和对噪声不敏感等特性而体现出极大的优势。也是由于这些原因,振荡性的脑电图变成了脑-机接口的应用中非常成功的设计之一。本文要介绍主要的基于脑电的脑-机接口中的两种,分别是稳态视觉诱发电位和基于运动本体感觉节律的脑-机接口。作者将详细的叙述该研究的生理背景、脑-机接口的参数,以及该系统的构造及信号处理的方法,并且会演示一些具有潜在应用价值的科研成果。     

A Generalizable Brain-Computer Interface (BCI) Using Machine Learning for Feature Discovery

This work describes a generalized method for classifying motor-related neural signals for a brain-computer interface (BCI), based on a stochastic machine learning method. The method differs from the various feature extraction and selection techniques employed in many other BCI systems. The classifier does not use extensive a-priori information, resulting in reduced reliance on highly specific domain knowledge. Instead of pre-defining features, the time-domain signal is input to a population of multi-layer perceptrons (MLPs) in order to perform a stochastic search for the best structure. The results showed that the average performance of the new algorithm outperformed other published methods using the Berlin BCI IV (2008) competition dataset and was comparable to the best results in the Berlin BCI II (2002–3) competition dataset. The new method was also applied to electroencephalography (EEG) data recorded from five subjects undertaking a hand squeeze task and demonstrated high levels of accuracy with a mean classification accuracy of 78.9% after five-fold cross-validation. Our new approach has been shown to give accurate results across different motor tasks and signal types as well as between subjects.     

神经工程与脑-机接口

神经工程是近年来在生物医学工程领域备受关注的学科发展新方向。它运用神经科学和工程学的方法来分析神经功能并为神经功能缺失与紊乱的修复提供新的解决问题的方案;而脑-机接口则是当前神经工程领域中最活跃的研究方向之一。脑-机接口是在脑与计算机或其他外部设备之间建立的直接的通信和交流通道。在脑-机接口系统中,具有特定模式的脑信号携带着受试者希望表达的意愿,计算机将接收到的脑信号转换成相应的控制命令,于是那些有运动障碍的残疾人就可以利用脑-机接口系统来实现与外界的交流与对外部设备的控制。在基于脑电信号的脑-机接口系统中,受试者产生的脑信号大致可以分为内源性（endogenous）和外源性（exogenous）两类。其中外源性的成分主要取决于外部物理刺激（视觉、听觉或触觉）的参数而与认知行为无关;而内源性成分则主要由认知行为产生而与外部的物理刺激无关。在许多情况下,脑－机接口中的瞬态诱发电位通常都同时包含着内源性和外源性两种成分。寻找新的脑－机接口模式使之能显著提升记录脑电信号中的内源性与外源性成分在脑－机接口研究中具有重要意义。本文中将介绍一种基于运动起始时刻（motion—onset）的新的脑-机接口实验范式。本文的最后还探讨了脑－机接口未来发展的趋势与展望。     

Regularized common spatial patterns with subject-to-subject transfer of EEG signals

In the context of brain-computer interface (BCI) system, the common spatial patterns (CSP) method has been used to extract discriminative spatial filters for the classification of electroencephalogram (EEG) signals. However, the classification performance of CSP typically deteriorates when a few training samples are collected from a new BCI user. In this paper, we propose an approach that maintains or improves the recognition accuracy of the system with only a small size of training data set. The proposed approach is formulated by regularizing the classical CSP technique with the strategy of transfer learning. Specifically, we incorporate into the CSP analysis inter-subject information involving the same task, by minimizing the difference between the inter-subject features. Experimental results on two data sets from BCI competitions show that the proposed approach greatly improves the classification performance over that of the conventional CSP method; the transformed variant proved to be successful in almost every case, based on a small number of available training samples.     

Decoding Individual Finger Movements from One Hand Using Human EEG Signals

Brain computer interface (BCI) is an assistive technology, which decodes neurophysiological signals generated by the human brain and translates them into control signals to control external devices, e.g., wheelchairs. One problem challenging noninvasive BCI technologies is the limited control dimensions from decoding movements of, mainly, large body parts, e.g., upper and lower limbs. It has been reported that complicated dexterous functions, i.e., finger movements, can be decoded in electrocorticography (ECoG) signals, while it remains unclear whether noninvasive electroencephalography (EEG) signals also have sufficient information to decode the same type of movements. Phenomena of broadband power increase and low-frequency-band power decrease were observed in EEG in the present study, when EEG power spectra were decomposed by a principal component analysis (PCA). These movement-related spectral structures and their changes caused by finger movements in EEG are consistent with observations in previous ECoG study, as well as the results from ECoG data in the present study. The average decoding accuracy of 77.11% over all subjects was obtained in classifying each pair of fingers from one hand using movement-related spectral changes as features to be decoded using a support vector machine (SVM) classifier. The average decoding accuracy in three epilepsy patients using ECoG data was 91.28% with the similarly obtained features and same classifier. Both decoding accuracies of EEG and ECoG are significantly higher than the empirical guessing level (51.26%) in all subjects (p<0.05). The present study suggests the similar movement-related spectral changes in EEG as in ECoG, and demonstrates the feasibility of discriminating finger movements from one hand using EEG. These findings are promising to facilitate the development of BCIs with rich control signals using noninvasive technologies.     

Extracting optimal tempo-spatial features using local discriminant bases and common spatial patterns for brain computer interfacing

Brain computer interfaces (BCI) provide a new approach to human computer communication, where the control is realised via performing mental tasks such as motor imagery (MI). In this study, we investigate a novel method to automatically segment electroencephalographic (EEG) data within a trial and extract features accordingly in order to improve the performance of MI data classification techniques. A new local discriminant bases (LDB) algorithm using common spatial patterns (CSP) projection as transform function is proposed for automatic trial segmentation. CSP is also used for feature extraction following trial segmentation. This new technique also allows to obtain a more accurate picture of the most relevant temporal–spatial points in the EEG during the MI. The results are compared with other standard temporal segmentation techniques such as sliding window and LDB based on the local cosine transform (LCT).