Application of competitive Hopfield neural network to brain-computer interface systems

We propose an unsupervised recognition system for single-trial classification of motor imagery (MI) electroencephalogram (EEG) data in this study. Competitive Hopfield neural network (CHNN) clustering is used for the discrimination of left and right MI EEG data posterior to selecting active segment and extracting fractal features in multi-scale. First, we use continuous wavelet transform (CWT) and Student's two-sample t-statistics to select the active segment in the time-frequency domain. The multiresolution fractal features are then extracted from wavelet data by means of modified fractal dimension. At last, CHNN clustering is adopted to recognize extracted features. Due to the characteristic of non-supervision, it is proper for CHNN to classify non-stationary EEG signals. The results indicate that CHNN achieves 81.9% in average classification accuracy in comparison with self-organizing map (SOM) and several popular supervised classifiers on six subjects from two data sets.     

Analyzing text recognition from tactually evoked EEG

Tactual exploration of objects produce specific patterns in the human brain and hence objects can be recognized by analyzing brain signals during tactile exploration. The present work aims at analyzing EEG signals online for recognition of embossed texts by tactual exploration. EEG signals are acquired from the parietal region over the somatosensory cortex of blindfolded healthy subjects while they tactually explored embossed texts, including symbols, numbers, and alphabets. Classifiers based on the principle of supervised learning are trained on the extracted EEG feature space, comprising three features, namely, adaptive autoregressive parameters, Hurst exponents, and power spectral density, to recognize the respective texts. The pre-trained classifiers are used to classify the EEG data to identify the texts online and the recognized text is displayed on the computer screen for communication. Online classifications of two, four, and six classes of embossed texts are achieved with overall average recognition rates of 76.62, 72.31, and 67.62% respectively and the computational time is less than 2 s in each case. The maximum information transfer rate and utility of the system performance over all experiments are 0.7187 and 2.0529 bits/s respectively. This work presents a study that shows the possibility to classify 3D letters using tactually evoked EEG. In future, it will help the BCI community to design stimuli for better tactile augmentation n also opens new directions of research to facilitate 3D letters for visually impaired persons. Further, 3D maps can be generated for aiding tactual BCI in teleoperation.     

神经工程与脑-机接口

神经工程是近年来在生物医学工程领域备受关注的学科发展新方向。它运用神经科学和工程学的方法来分析神经功能并为神经功能缺失与紊乱的修复提供新的解决问题的方案;而脑-机接口则是当前神经工程领域中最活跃的研究方向之一。脑-机接口是在脑与计算机或其他外部设备之间建立的直接的通信和交流通道。在脑-机接口系统中,具有特定模式的脑信号携带着受试者希望表达的意愿,计算机将接收到的脑信号转换成相应的控制命令,于是那些有运动障碍的残疾人就可以利用脑-机接口系统来实现与外界的交流与对外部设备的控制。在基于脑电信号的脑-机接口系统中,受试者产生的脑信号大致可以分为内源性（endogenous）和外源性（exogenous）两类。其中外源性的成分主要取决于外部物理刺激（视觉、听觉或触觉）的参数而与认知行为无关;而内源性成分则主要由认知行为产生而与外部的物理刺激无关。在许多情况下,脑－机接口中的瞬态诱发电位通常都同时包含着内源性和外源性两种成分。寻找新的脑－机接口模式使之能显著提升记录脑电信号中的内源性与外源性成分在脑－机接口研究中具有重要意义。本文中将介绍一种基于运动起始时刻（motion—onset）的新的脑-机接口实验范式。本文的最后还探讨了脑－机接口未来发展的趋势与展望。     

Feature extraction and recognition of epileptiform activity in EEG by combining PCA with ApEn

This paper proposes a new method for feature extraction and recognition of epileptiform activity in EEG signals. The method improves feature extraction speed of epileptiform activity without reducing recognition rate. Firstly, Principal component analysis (PCA) is applied to the original EEG for dimension reduction and to the decorrelation of epileptic EEG and normal EEG. Then discrete wavelet transform (DWT) combined with approximate entropy (ApEn) is performed on epileptic EEG and normal EEG, respectively. At last, Neyman–Pearson criteria are applied to classify epileptic EEG and normal ones. The main procedure is that the principle component of EEG after PCA is decomposed into several sub-band signals using DWT, and ApEn algorithm is applied to the sub-band signals at different wavelet scales. Distinct difference is found between the ApEn values of epileptic and normal EEG. The method allows recognition of epileptiform activities and discriminates them from the normal EEG. The algorithm performs well at epileptiform activity recognition in the clinic EEG data and offers a flexible tool that is intended to be generalized to the simultaneous recognition of many waveforms in EEG.     

Analysis and classification of speech imagery EEG for BCI

Electroencephalogram (EEG) is generally used in brain–computer interface (BCI), including motor imagery, mental task, steady-state evoked potentials (SSEPs) and P300. In order to complement existing motor-based control paradigms, this paper proposed a novel imagery mode: speech imagery. Chinese characters are monosyllabic and one Chinese character can express one meaning. Thus, eight Chinese subjects were required to read two Chinese characters in mind in this experiment. There were different shapes, pronunciations and meanings between two Chinese characters. Feature vectors of EEG signals were extracted by common spatial patterns (CSP), and then these vectors were classified by support vector machine (SVM). The accuracy between two characters was not superior. However, it was still effective to distinguish whether subjects were reading one character in mind, and the accuracies were between 73.65% and 95.76%. The results were better than vowel speech imagery, and they were suitable for asynchronous BCI. BCI systems will be also extended from motor imagery to combine motor imagery and speech imagery in the future.     

Individually Adapted Imagery Improves Brain-Computer Interface Performance in End-Users with Disability

Brain-computer interfaces (BCIs) translate oscillatory electroencephalogram (EEG) patterns into action. Different mental activities modulate spontaneous EEG rhythms in various ways. Non-stationarity and inherent variability of EEG signals, however, make reliable recognition of modulated EEG patterns challenging. Able-bodied individuals who use a BCI for the first time achieve - on average - binary classification performance of about 75%. Performance in users with central nervous system (CNS) tissue damage is typically lower. User training generally enhances reliability of EEG pattern generation and thus also robustness of pattern recognition. In this study, we investigated the impact of mental tasks on binary classification performance in BCI users with central nervous system (CNS) tissue damage such as persons with stroke or spinal cord injury (SCI). Motor imagery (MI), that is the kinesthetic imagination of movement (e.g. squeezing a rubber ball with the right hand), is the "gold standard" and mainly used to modulate EEG patterns. Based on our recent results in able-bodied users, we hypothesized that pair-wise combination of "brain-teaser" (e.g. mental subtraction and mental word association) and "dynamic imagery" (e.g. hand and feet MI) tasks significantly increases classification performance of induced EEG patterns in the selected end-user group. Within-day (How stable is the classification within a day?) and between-day (How well does a model trained on day one perform on unseen data of day two?) analysis of variability of mental task pair classification in nine individuals confirmed the hypothesis. We found that the use of the classical MI task pair hand vs. feed leads to significantly lower classification accuracy - in average up to 15% less - in most users with stroke or SCI. User-specific selection of task pairs was again essential to enhance performance. We expect that the gained evidence will significantly contribute to make imagery-based BCI technology become accessible to a larger population of users including individuals with special needs due to CNS damage.     

Brain–computer interface: The first experience of clinical use in Russia

Motor imagery can stimulate the same neuroplastic mechanisms of the brain as their actual execution. The motor imagery can be controlled via the brain–computer interface (BCI), which transforms the EEG signals of the brain appearing during the motor imagery into commands for the external device. The results of the two-stage study of the application of a non-invasive BCI for the rehabilitation of patients with marked hemiparesis resulted from a local brain injury. We have shown that the learning to manage the BCI does not depend on the duration of disease, localization of the damaged site, and the severity of neurological deficit. The results of the first stage of the study carried out in a group of 36 patients showed that the rehabilitation therapy was more effective in the group that was trained to manage the BCI (the ARAT score improved from 1 [0; 2] to 5 [0; 16], p = 0.012 in the BCI group; no significant improvement was detected in the control group). In the second phase of the study, 19 patients participated in the testing of a BCI–exoskeleton system. Rehabilitation based on this technology led to an improvement of the motor function of an arm from 2 [0; 37] to 4 [1; 45.5], p = 0.005, according to the ARAT scale, and from 72 [63; 110] to 79 [68; 115], p = 0.005, according to the Fugl-Meyer scale.     

Regularized common spatial patterns with subject-to-subject transfer of EEG signals

In the context of brain-computer interface (BCI) system, the common spatial patterns (CSP) method has been used to extract discriminative spatial filters for the classification of electroencephalogram (EEG) signals. However, the classification performance of CSP typically deteriorates when a few training samples are collected from a new BCI user. In this paper, we propose an approach that maintains or improves the recognition accuracy of the system with only a small size of training data set. The proposed approach is formulated by regularizing the classical CSP technique with the strategy of transfer learning. Specifically, we incorporate into the CSP analysis inter-subject information involving the same task, by minimizing the difference between the inter-subject features. Experimental results on two data sets from BCI competitions show that the proposed approach greatly improves the classification performance over that of the conventional CSP method; the transformed variant proved to be successful in almost every case, based on a small number of available training samples.     

基于节律性脑电信号的脑-机接口

脑-机接口系统是一个不依靠外周神经和肌肉组织等而实现大脑和外界装置之间直接的交流和控制的通道。它为那些运动障碍的残疾人表达自己的意愿和实现对外部设备的控制提供了一种新的强大的技术支持。基于脑电的脑-机接口作为一种非侵入型的技术引起了该领域很多人的关注。基于脑电的脑-机接口采用了很多种类型的脑电信号。其中,振荡性的脑电图由于有较高的幅值和对噪声不敏感等特性而体现出极大的优势。也是由于这些原因,振荡性的脑电图变成了脑-机接口的应用中非常成功的设计之一。本文要介绍主要的基于脑电的脑-机接口中的两种,分别是稳态视觉诱发电位和基于运动本体感觉节律的脑-机接口。作者将详细的叙述该研究的生理背景、脑-机接口的参数,以及该系统的构造及信号处理的方法,并且会演示一些具有潜在应用价值的科研成果。     

Hybrid EEG-fNIRS Asynchronous Brain-Computer Interface for Multiple Motor Tasks

Non-invasive Brain-Computer Interfaces (BCI) have demonstrated great promise for neuroprosthetics and assistive devices. Here we aim to investigate methods to combine Electroencephalography (EEG) and functional Near-Infrared Spectroscopy (fNIRS) in an asynchronous Sensory Motor rhythm (SMR)-based BCI. We attempted to classify 4 different executed movements, namely, Right-Arm—Left-Arm—Right-Hand—Left-Hand tasks. Previous studies demonstrated the benefit of EEG-fNIRS combination. However, since normally fNIRS hemodynamic response shows a long delay, we investigated new features, involving slope indicators, in order to immediately detect changes in the signals. Moreover, Common Spatial Patterns (CSPs) have been applied to both EEG and fNIRS signals. 15 healthy subjects took part in the experiments and since 25 trials per class were available, CSPs have been regularized with information from the entire population of participants and optimized using genetic algorithms. The different features have been compared in terms of performance and the dynamic accuracy over trials shows that the introduced methods diminish the fNIRS delay in the detection of changes.     

Robust Brain-computer Interface for virtual Keyboard (RoBIK): Project results

Brain-Computer Interface (BCI) is a technology that translates the brain electrical activity into a command for a device such as a robotic arm, a wheelchair or a spelling device. BCIs have long been described as an assistive technology for severely disabled patients because they completely bypass the need for muscular activity. The clinical reality is however dramatically different and most patients who use BCIs today are doing so as part of constraining clinical trials. To achieve the technological transfer from bench to bedside, BCI must gain ease of use and robustness of both measure (electroencephalography [EEG]) and interface (signal processing and applications). The Robust Brain-computer Interface for virtual Keyboard (RoBIK) project aimed at the development of a BCI system for communication that could be used on a daily basis by patients without the help of a trained team of researchers. To guide further developments clinicians first assessed patients’ needs. The prototype subsequently developed consisted in a 14 felt-pad electrodes EEG headset sampling at 256 Hz by an electronic component capable of transmitting signals wirelessly. The application was a virtual keyboard generating a novel stimulation paradigm to elicit P300 Evoked Related Potentials (ERPs) for communication. Raw EEG signals were treated with OpenViBE open-source software including novel signal processing and stimulation techniques.     

Brain-computer interfaces for control of neuroprostheses: from synchronous to asynchronous mode of operation.

Transferring a brain-computer interface (BCI) from the laboratory environment into real world applications is directly related to the problem of identifying user intentions from brain signals without any additional information in real time. From the perspective of signal processing, the BCI has to have an uncued or asynchronous design. Based on the results of two clinical applications, where 'thought' control of neuroprostheses based on movement imagery in tetraplegic patients with a high spinal cord injury has been established, the general steps from a synchronous or cue-guided BCI to an internally driven asynchronous brain-switch are discussed. The future potential of BCI methods for various control purposes, especially for functional rehabilitation of tetraplegics using neuroprosthetics, is outlined.     

Classification of Motor Imagery EEG Based on Time-Domain and Frequency-Domain Dual-Stream Convolutional Neural Network

Background and objectiveAn important task of the brain-computer interface (BCI) of motor imagery is to extract effective time-domain features, frequency-domain features or time-frequency domain features from the raw electroencephalogram (EEG) signals for classification of motor imagery. However, choosing an appropriate method to combine time domain and frequency domain features to improve the performance of motor imagery recognition is still a research hotspot.MethodsIn order to fully extract and utilize the time-domain and frequency-domain features of EEG in classification tasks, this paper proposed a novel dual-stream convolutional neural network (DCNN), which can use time domain signal and frequency domain signal as the inputs, and the extracted time-domain features and frequency-domain features are fused by linear weighting for classification training. Furthermore, the weight can be learned by the DCNN automatically.ResultsThe experiments based on BCI competition II dataset III and BCI competition IV dataset 2a showed that the model proposed by this study has better performance than other conventional methods. The model used time-frequency signal as the inputs had better performance than the model only used time-domain signals or frequency-domain signals. The accuracy of classification was improved for each subject compared with the models only used one signals as the inputs.ConclusionsFurther analysis shown that the fusion weight of different subject is specifically, adjusting the weight coefficient automatically is helpful to improve the classification accuracy.     

Selection of relevant features for EEG signal classification of schizophrenic patients

In this paper, EEG signals of 20 schizophrenic patients and 20 age-matched control participants are analyzed with the objective of determining the more informative channels and finally distinguishing the two groups. For each case, 22 channels of EEG were recorded. A two-stage feature selection algorithm is designed, such that, the more informative channels are first selected to enhance the discriminative information. Two methods, bidirectional search and plus-L minus-R (LRS) techniques are employed to select these informative channels. The interesting point is that most of selected channels are located in the temporal lobes (containing the limbic system) that confirm the neuro-phychological differences in these areas between the schizophrenic and normal participants. After channel selection, genetic algorithm (GA) is employed to select the best features from the selected channels. In this case, in addition to elimination of the less informative channels, the redundant and less discriminant features are also eliminated. A computationally fast algorithm with excellent classification results is obtained. Implementation of this efficient approach involves several features including autoregressive (AR) model parameters, band power, fractal dimension and wavelet energy. To test the performance of the final subset of features, classifiers including linear discriminant analysis (LDA) and support vector machine (SVM) are employed to classify the reduced feature set of the two groups. Using the bidirectional search for channel selection, a classification accuracy of 84.62% and 99.38% is obtained for LDA and SVM, respectively. Using the LRS technique for channel selection, a classification accuracy of 88.23% and 99.54% is also obtained for LDA and SVM, respectively. Finally, the results are compared and contrasted with two well-known methods namely, the single-stage feature selection (evolutionary feature selection) and principal component analysis (PCA)-based feature selection. The results show improved accuracy of classification in relatively low computational time with the two-stage feature selection.     

Estimating the Intended Sound Direction of the User: Toward an Auditory Brain-Computer Interface Using Out-of-Head Sound Localization

The auditory Brain-Computer Interface (BCI) using electroencephalograms (EEG) is a subject of intensive study. As a cue, auditory BCIs can deal with many of the characteristics of stimuli such as tone, pitch, and voices. Spatial information on auditory stimuli also provides useful information for a BCI. However, in a portable system, virtual auditory stimuli have to be presented spatially through earphones or headphones, instead of loudspeakers. We investigated the possibility of an auditory BCI using the out-of-head sound localization technique, which enables us to present virtual auditory stimuli to users from any direction, through earphones. The feasibility of a BCI using this technique was evaluated in an EEG oddball experiment and offline analysis. A virtual auditory stimulus was presented to the subject from one of six directions. Using a support vector machine, we were able to classify whether the subject attended the direction of a presented stimulus from EEG signals. The mean accuracy across subjects was 70.0% in the single-trial classification. When we used trial-averaged EEG signals as inputs to the classifier, the mean accuracy across seven subjects reached 89.5% (for 10-trial averaging). Further analysis showed that the P300 event-related potential responses from 200 to 500 ms in central and posterior regions of the brain contributed to the classification. In comparison with the results obtained from a loudspeaker experiment, we confirmed that stimulus presentation by out-of-head sound localization achieved similar event-related potential responses and classification performances. These results suggest that out-of-head sound localization enables us to provide a high-performance and loudspeaker-less portable BCI system.     

基于多特征融合的多分类运动想象脑电信号识别研究

针对目前多分类运动想象脑电识别存在特征提取单一、分类准确率低等问题,提出一种多特征融合的四分类运动想象脑电识别方法来提高识别率。对预处理后的脑电信号分别使用希尔伯特-黄变换、一对多共空间模式、近似熵、模糊熵、样本熵提取结合时频—空域—非线性动力学的初始特征向量,用主成分分析降维,最后使用粒子群优化支持向量机分类。该算法通过对国际标准数据集BCI2005 Data set IIIa中的k3b受试者数据经MATLAB仿真处理后获得93.30%的识别率,均高于单一特征和其它组合特征下的识别率。分别对四名实验者实验采集运动想象脑电数据,使用本研究提出的方法处理获得了72.96%的平均识别率。结果表明多特征融合的特征提取方法能更好的表征运动想象脑电信号,使用粒子群支持向量机可取得较高的识别准确率,为人脑的认知活动提供了一种新的识别方法。     

Brain-Computer Interface Speller System for Alternative Communication: A Review

Brain-computer interface (BCI) speller is a system that provides an alternative communication for the disable people. The brain wave is translated into machine command through a BCI speller which can be used as a communication medium for the patients to express their thought without any motor movement. A BCI speller aims to spell characters by using the electroencephalogram (EEG) signal. Several types of BCI spellers are available based on the EEG signal. A standard BCI speller system consists of the following elements: BCI speller paradigm, data acquisition system and signal processing algorithms. In this work, a systematic review is provided on the BCI speller system and it includes speller paradigms, feature extraction, feature optimization and classification techniques for BCI speller. The advantages and limitations of different speller paradigm and machine learning algorithms are discussed in this article. Also, the future research directions are discussed which can overcome the limitations of present state-of-the-art techniques for BCI speller.     

Principles of neurorehabilitation based on the brain-computer interface and biologically adequate control of the exoskeleton

The neurophysiological prerequisites for the development and operation of the brain-computer interfaces (BCI) that allow cerebral electrical signals alone to control external technical devices are considered. A BCI based on the discrimination of the EEG patterns related to imagery of extremity movements is described. The possibility of the rehabilitation of patients with motor disorders by means of the BCI based on motor imagery and the exoskeleton controlled by it is discussed.     

EEG Classification of Different Imaginary Movements within the Same Limb

The task of discriminating the motor imagery of different movements within the same limb using electroencephalography (EEG) signals is challenging because these imaginary movements have close spatial representations on the motor cortex area. There is, however, a pressing need to succeed in this task. The reason is that the ability to classify different same-limb imaginary movements could increase the number of control dimensions of a brain-computer interface (BCI). In this paper, we propose a 3-class BCI system that discriminates EEG signals corresponding to rest, imaginary grasp movements, and imaginary elbow movements. Besides, the differences between simple motor imagery and goal-oriented motor imagery in terms of their topographical distributions and classification accuracies are also being investigated. To the best of our knowledge, both problems have not been explored in the literature. Based on the EEG data recorded from 12 able-bodied individuals, we have demonstrated that same-limb motor imagery classification is possible. For the binary classification of imaginary grasp and elbow (goal-oriented) movements, the average accuracy achieved is 66.9%. For the 3-class problem of discriminating rest against imaginary grasp and elbow movements, the average classification accuracy achieved is 60.7%, which is greater than the random classification accuracy of 33.3%. Our results also show that goal-oriented imaginary elbow movements lead to a better classification performance compared to simple imaginary elbow movements. This proposed BCI system could potentially be used in controlling a robotic rehabilitation system, which can assist stroke patients in performing task-specific exercises.     

A Deep Learning Architecture for P300 Detection with Brain-Computer Interface Application

In this paper, a brain-computer interface (BCI) system for character recognition is proposed based on the P300 signal. A P300 speller is used to spell the word or character without any muscle movement. P300 detection is the first step to detect the character from the electroencephalogram (EEG) signal. The character is recognized from the detected P300 signal. In this paper, sparse autoencoder (SAE) and stacked sparse autoencoder (SSAE) based feature extraction methods are proposed for P300 detection. This work also proposes a fusion of deep-features with the temporal features for P300 detection. A SSAE technique extracts high-level information about input data. The combination of SSAE features with the temporal features provides abstract and temporal information about the signal. An ensemble of weighted artificial neural network (EWANN) is proposed for P300 detection to minimize the variation among different classifiers. To provide more importance to the good classifier for final classification, a higher weightage is assigned to the better performing classifier. These weights are calculated from the cross-validation test. The model is tested on two different publicly available datasets, and the proposed method provides better or comparable character recognition performance than the state-of-the-art methods.