Localization of brain electrical activity sources and hemodynamic activity foci during motor imagery

The sources of brain activity that make the maximum contribution to EEG patterns corresponding to motor imagery have been studied. The accuracy of their classification determines the efficiency of brain-computer interface (BCI) for controlling external technical devices directly by brain signals, without the involvement of muscle activity. Brain activity sources are identified by independent component analysis. The independent components providing the maximum BCI classification accuracy are considered relevant for the motor imagery task. The two most relevant sources exhibit clearly marked event-related desynchronization and synchronization of the μ-rhythm during the imagery of contra- and ipsilateral hand movements. These sources were localized by solving the inverse EEG problem with due consideration for individual geometry of the brain and its covers, as determined by magnetic resonance imaging. Each of the sources was shown to be localized in the 3a area of the primary somatosensory cortex corresponding to proprioceptive sensitivity of the contralateral hand. Their positions were close to the foci of BOLD activity obtained by fMRI.     

Classification of Motor Imagery EEG Based on Time-Domain and Frequency-Domain Dual-Stream Convolutional Neural Network

Background and objectiveAn important task of the brain-computer interface (BCI) of motor imagery is to extract effective time-domain features, frequency-domain features or time-frequency domain features from the raw electroencephalogram (EEG) signals for classification of motor imagery. However, choosing an appropriate method to combine time domain and frequency domain features to improve the performance of motor imagery recognition is still a research hotspot.MethodsIn order to fully extract and utilize the time-domain and frequency-domain features of EEG in classification tasks, this paper proposed a novel dual-stream convolutional neural network (DCNN), which can use time domain signal and frequency domain signal as the inputs, and the extracted time-domain features and frequency-domain features are fused by linear weighting for classification training. Furthermore, the weight can be learned by the DCNN automatically.ResultsThe experiments based on BCI competition II dataset III and BCI competition IV dataset 2a showed that the model proposed by this study has better performance than other conventional methods. The model used time-frequency signal as the inputs had better performance than the model only used time-domain signals or frequency-domain signals. The accuracy of classification was improved for each subject compared with the models only used one signals as the inputs.ConclusionsFurther analysis shown that the fusion weight of different subject is specifically, adjusting the weight coefficient automatically is helpful to improve the classification accuracy.     

The Efficiency of the Brain-Computer Interfaces Based on Motor Imagery with Tactile and Visual Feedback

In this study we compared tactile and visual feedbacks for the motor imagery-based brain–computer interface (BCI) in five healthy subjects. A vertical green bar from the center of the fixing cross to the edge of the screen was used as visual feedback. Vibration motors that were placed on the forearms of the right and the left hands and on the back of the subject’s neck were used as tactile feedback. A vibration signal was used to confirm the correct classification of the EEG patterns of the motor imagery of right and left hand movements and the rest task. The accuracy of recognition in the classification of the three states (right hand movement, left hand movement, and rest) in the BCI without feedback exceeded the random level (33% for the three states) for all the subjects and was rather high (67.8% ± 13.4% (mean ± standard deviation)). Including the visual and tactile feedback in the BCI did not significantly change the mean accuracy of recognition of mental states for all the subjects (70.5% ± 14.8% for the visual feedback and 65.9% ± 12.4% for the tactile feedback). The analysis of the dynamics of the movement imagery skill in BCI users with the tactile and visual feedback showed no significant differences between these types of feedback. Thus, it has been found that the tactile feedback can be used in the motor imagery-based BCI instead of the commonly used visual feedback, which greatly expands the possibilities of the practical application of the BCI.     

Translation of EEG spatial filters from resting to motor imagery using independent component analysis

Electroencephalogram (EEG)-based brain-computer interfaces (BCIs) often use spatial filters to improve signal-to-noise ratio of task-related EEG activities. To obtain robust spatial filters, large amounts of labeled data, which are often expensive and labor-intensive to obtain, need to be collected in a training procedure before online BCI control. Several studies have recently developed zero-training methods using a session-to-session scenario in order to alleviate this problem. To our knowledge, a state-to-state translation, which applies spatial filters derived from one state to another, has never been reported. This study proposes a state-to-state, zero-training method to construct spatial filters for extracting EEG changes induced by motor imagery. Independent component analysis (ICA) was separately applied to the multi-channel EEG in the resting and the motor imagery states to obtain motor-related spatial filters. The resultant spatial filters were then applied to single-trial EEG to differentiate left- and right-hand imagery movements. On a motor imagery dataset collected from nine subjects, comparable classification accuracies were obtained by using ICA-based spatial filters derived from the two states (motor imagery: 87.0%, resting: 85.9%), which were both significantly higher than the accuracy achieved by using monopolar scalp EEG data (80.4%). The proposed method considerably increases the practicality of BCI systems in real-world environments because it is less sensitive to electrode misalignment across different sessions or days and does not require annotated pilot data to derive spatial filters.     

Improving the Performance of an EEG-Based Motor Imagery Brain Computer Interface Using Task Evoked Changes in Pupil Diameter

For individuals with high degrees of motor disability or locked-in syndrome, it is impractical or impossible to use mechanical switches to interact with electronic devices. Brain computer interfaces (BCIs) can use motor imagery to detect interaction intention from users but lack the accuracy of mechanical switches. Hence, there exists a strong need to improve the accuracy of EEG-based motor imagery BCIs attempting to implement an on/off switch. Here, we investigate how monitoring the pupil diameter of a person as a psycho-physiological parameter in addition to traditional EEG channels can improve the classification accuracy of a switch-like BCI. We have recently noticed in our lab (work not yet published) how motor imagery is associated with increases in pupil diameter when compared to a control rest condition. The pupil diameter parameter is easily accessible through video oculography since most gaze tracking systems report pupil diameter invariant to head position. We performed a user study with 30 participants using a typical EEG based motor imagery BCI. We used common spatial patterns to separate motor imagery, signaling movement intention, from a rest control condition. By monitoring the pupil diameter of the user and using this parameter as an additional feature, we show that the performance of the classifier trying to discriminate motor imagery from a control condition improves over the traditional approach using just EEG derived features. Given the limitations of EEG to construct highly robust and reliable BCIs, we postulate that multi-modal approaches, such as the one presented here that monitor several psycho-physiological parameters, can be a successful strategy in making BCIs more accurate and less vulnerable to constraints such as requirements for long training sessions or high signal to noise ratio of electrode channels.     

基于多特征融合的多分类运动想象脑电信号识别研究

针对目前多分类运动想象脑电识别存在特征提取单一、分类准确率低等问题,提出一种多特征融合的四分类运动想象脑电识别方法来提高识别率。对预处理后的脑电信号分别使用希尔伯特-黄变换、一对多共空间模式、近似熵、模糊熵、样本熵提取结合时频—空域—非线性动力学的初始特征向量,用主成分分析降维,最后使用粒子群优化支持向量机分类。该算法通过对国际标准数据集BCI2005 Data set IIIa中的k3b受试者数据经MATLAB仿真处理后获得93.30%的识别率,均高于单一特征和其它组合特征下的识别率。分别对四名实验者实验采集运动想象脑电数据,使用本研究提出的方法处理获得了72.96%的平均识别率。结果表明多特征融合的特征提取方法能更好的表征运动想象脑电信号,使用粒子群支持向量机可取得较高的识别准确率,为人脑的认知活动提供了一种新的识别方法。     

Principles of neurorehabilitation based on the brain-computer interface and biologically adequate control of the exoskeleton

The neurophysiological prerequisites for the development and operation of the brain-computer interfaces (BCI) that allow cerebral electrical signals alone to control external technical devices are considered. A BCI based on the discrimination of the EEG patterns related to imagery of extremity movements is described. The possibility of the rehabilitation of patients with motor disorders by means of the BCI based on motor imagery and the exoskeleton controlled by it is discussed.     

Analysis and classification of speech imagery EEG for BCI

Electroencephalogram (EEG) is generally used in brain–computer interface (BCI), including motor imagery, mental task, steady-state evoked potentials (SSEPs) and P300. In order to complement existing motor-based control paradigms, this paper proposed a novel imagery mode: speech imagery. Chinese characters are monosyllabic and one Chinese character can express one meaning. Thus, eight Chinese subjects were required to read two Chinese characters in mind in this experiment. There were different shapes, pronunciations and meanings between two Chinese characters. Feature vectors of EEG signals were extracted by common spatial patterns (CSP), and then these vectors were classified by support vector machine (SVM). The accuracy between two characters was not superior. However, it was still effective to distinguish whether subjects were reading one character in mind, and the accuracies were between 73.65% and 95.76%. The results were better than vowel speech imagery, and they were suitable for asynchronous BCI. BCI systems will be also extended from motor imagery to combine motor imagery and speech imagery in the future.     

Proprioceptive Feedback and Brain Computer Interface (BCI) Based Neuroprostheses

Brain computer interface (BCI) technology has been proposed for motor neurorehabilitation, motor replacement and assistive technologies. It is an open question whether proprioceptive feedback affects the regulation of brain oscillations and therefore BCI control. We developed a BCI coupled on-line with a robotic hand exoskeleton for flexing and extending the fingers. 24 healthy participants performed five different tasks of closing and opening the hand: (1) motor imagery of the hand movement without any overt movement and without feedback, (2) motor imagery with movement as online feedback (participants see and feel their hand, with the exoskeleton moving according to their brain signals, (3) passive (the orthosis passively opens and closes the hand without imagery) and (4) active (overt) movement of the hand and rest. Performance was defined as the difference in power of the sensorimotor rhythm during motor task and rest and calculated offline for different tasks. Participants were divided in three groups depending on the feedback receiving during task 2 (the other tasks were the same for all participants). Group 1 (n = 9) received contingent positive feedback (participants'' sensorimotor rhythm (SMR) desynchronization was directly linked to hand orthosis movements), group 2 (n = 8) contingent “negative” feedback (participants'' sensorimotor rhythm synchronization was directly linked to hand orthosis movements) and group 3 (n = 7) sham feedback (no link between brain oscillations and orthosis movements). We observed that proprioceptive feedback (feeling and seeing hand movements) improved BCI performance significantly. Furthermore, in the contingent positive group only a significant motor learning effect was observed enhancing SMR desynchronization during motor imagery without feedback in time. Furthermore, we observed a significantly stronger SMR desynchronization in the contingent positive group compared to the other groups during active and passive movements. To summarize, we demonstrated that the use of contingent positive proprioceptive feedback BCI enhanced SMR desynchronization during motor tasks.     

Bayesian classifier for brain-computer interface based on mental representation of movements

This paper proposes Bayesian approach to classification of EEG patterns on the basis of imaginary movements of extremities based on analysis ofcovariance matrices of native EEG recordings. An efficacy of a Brain-Computer Interface (BCI) based on the proposed classifier is evaluated. Bayesian classifier is shown to be competitive with the MCSP (Multiclass Common Spatial Patterns) classifier known from the literature as one of the efficient variant for BCI implementation. The influence of eye movement and blinking artifacts on the BCI performance is investigated. It is shown that the presence of such artifacts does not affect the classification accuracy.     

Online Classification of Multiple Motor Imagery Tasks Using Filter Bank Based Maximum-a-Posteriori Common Spatial Pattern Filters

ObjectiveThe main objective of this paper is to propose a novel technique, called filter bank maximum a-posteriori common spatial pattern (FB-MAP-CSP) algorithm, for online classification of multiple motor imagery activities using electroencephalography (EEG) signals. The proposed technique addresses the overfitting issue of CSP in addition to utilizing the spectral information of EEG signals inside the framework of filter banks while extending it to more than two conditions.Materials and methodsThe classification of motor imagery signals is based upon the detection of event-related de-synchronization (ERD) phenomena in the μ and β rhythms of EEG signals. Accordingly, two modifications in the existing MAP-CSP technique are presented: (i) The (pre-processed) EEG signals are spectrally filtered by a bank of filters lying in the μ and β brainwave frequency range, (ii) the framework of MAP-CSP is extended to deal with multiple (more than two) motor imagery tasks classification and the spatial filters thus obtained are calculated for each sub-band, separately. Subsequently, the most imperative features over all sub-bands are selected and un-regularized linear discriminant analysis is employed for classification of multiple motor imagery tasks.ResultsPublicly available dataset (BCI Competition IV Dataset I) is used to validate the proposed method i.e. FB-MAP-CSP. The results show that the proposed method yields superior classification results, in addition to be computationally more efficient in the case of online implementation, as compared to the conventional CSP based techniques and its variants for multiclass motor imagery classification.ConclusionThe proposed FB-MAP-CSP algorithm is found to be a potential / superior method for classifying multi-condition motor imagery EEG signals in comparison to FBCSP based techniques.     

A Neural Network-Based Optimal Spatial Filter Design Method for Motor Imagery Classification

In this study, a novel spatial filter design method is introduced. Spatial filtering is an important processing step for feature extraction in motor imagery-based brain-computer interfaces. This paper introduces a new motor imagery signal classification method combined with spatial filter optimization. We simultaneously train the spatial filter and the classifier using a neural network approach. The proposed spatial filter network (SFN) is composed of two layers: a spatial filtering layer and a classifier layer. These two layers are linked to each other with non-linear mapping functions. The proposed method addresses two shortcomings of the common spatial patterns (CSP) algorithm. First, CSP aims to maximize the between-classes variance while ignoring the minimization of within-classes variances. Consequently, the features obtained using the CSP method may have large within-classes variances. Second, the maximizing optimization function of CSP increases the classification accuracy indirectly because an independent classifier is used after the CSP method. With SFN, we aimed to maximize the between-classes variance while minimizing within-classes variances and simultaneously optimizing the spatial filter and the classifier. To classify motor imagery EEG signals, we modified the well-known feed-forward structure and derived forward and backward equations that correspond to the proposed structure. We tested our algorithm on simple toy data. Then, we compared the SFN with conventional CSP and its multi-class version, called one-versus-rest CSP, on two data sets from BCI competition III. The evaluation results demonstrate that SFN is a good alternative for classifying motor imagery EEG signals with increased classification accuracy.     

Individually Adapted Imagery Improves Brain-Computer Interface Performance in End-Users with Disability

Brain-computer interfaces (BCIs) translate oscillatory electroencephalogram (EEG) patterns into action. Different mental activities modulate spontaneous EEG rhythms in various ways. Non-stationarity and inherent variability of EEG signals, however, make reliable recognition of modulated EEG patterns challenging. Able-bodied individuals who use a BCI for the first time achieve - on average - binary classification performance of about 75%. Performance in users with central nervous system (CNS) tissue damage is typically lower. User training generally enhances reliability of EEG pattern generation and thus also robustness of pattern recognition. In this study, we investigated the impact of mental tasks on binary classification performance in BCI users with central nervous system (CNS) tissue damage such as persons with stroke or spinal cord injury (SCI). Motor imagery (MI), that is the kinesthetic imagination of movement (e.g. squeezing a rubber ball with the right hand), is the "gold standard" and mainly used to modulate EEG patterns. Based on our recent results in able-bodied users, we hypothesized that pair-wise combination of "brain-teaser" (e.g. mental subtraction and mental word association) and "dynamic imagery" (e.g. hand and feet MI) tasks significantly increases classification performance of induced EEG patterns in the selected end-user group. Within-day (How stable is the classification within a day?) and between-day (How well does a model trained on day one perform on unseen data of day two?) analysis of variability of mental task pair classification in nine individuals confirmed the hypothesis. We found that the use of the classical MI task pair hand vs. feed leads to significantly lower classification accuracy - in average up to 15% less - in most users with stroke or SCI. User-specific selection of task pairs was again essential to enhance performance. We expect that the gained evidence will significantly contribute to make imagery-based BCI technology become accessible to a larger population of users including individuals with special needs due to CNS damage.     

The changes in the hemodynamic activity of the brain during motor imagery training with the use of brain-computer interface

With the use of functional MRI (fMRI), we studied the changes in brain hemodynamic activity of healthy subjects during motor imagery training with the use brain-computer interface (BCI), which is based on the recognition of EEG patterns of imagined movements. ANOVA dispersion analysis showed there are 14 areas of the brain where statistically significant changes were registered. Detailed analysis of the activity in these areas before and after training (Student’s and Mann-Whitney tests) showed that the real amount of such areas is five; these are Brodmann areas 44 and 45, insula, middle frontal gyrus and anterior cingulate gyrus. We suggest that these changed are caused by the formation of memory traces of those brain activity patterns which are most accurately recognized by BCI classifiers as correspondent with limb movements imagery. We also observed a tendency of increase in the activity of motor imagery after training. The hemodynamic activity in all these 14 areas during real movements was either approximately the same or significantly higher than during motor imagery; activity during imagined leg movements was higher than that during imagines arm movements, except for the areas of representation of arms.     

Decoding Individual Finger Movements from One Hand Using Human EEG Signals

Brain computer interface (BCI) is an assistive technology, which decodes neurophysiological signals generated by the human brain and translates them into control signals to control external devices, e.g., wheelchairs. One problem challenging noninvasive BCI technologies is the limited control dimensions from decoding movements of, mainly, large body parts, e.g., upper and lower limbs. It has been reported that complicated dexterous functions, i.e., finger movements, can be decoded in electrocorticography (ECoG) signals, while it remains unclear whether noninvasive electroencephalography (EEG) signals also have sufficient information to decode the same type of movements. Phenomena of broadband power increase and low-frequency-band power decrease were observed in EEG in the present study, when EEG power spectra were decomposed by a principal component analysis (PCA). These movement-related spectral structures and their changes caused by finger movements in EEG are consistent with observations in previous ECoG study, as well as the results from ECoG data in the present study. The average decoding accuracy of 77.11% over all subjects was obtained in classifying each pair of fingers from one hand using movement-related spectral changes as features to be decoded using a support vector machine (SVM) classifier. The average decoding accuracy in three epilepsy patients using ECoG data was 91.28% with the similarly obtained features and same classifier. Both decoding accuracies of EEG and ECoG are significantly higher than the empirical guessing level (51.26%) in all subjects (p<0.05). The present study suggests the similar movement-related spectral changes in EEG as in ECoG, and demonstrates the feasibility of discriminating finger movements from one hand using EEG. These findings are promising to facilitate the development of BCIs with rich control signals using noninvasive technologies.     

A Comparative Approach for MI-Based EEG Signals Classification Using Energy,Power and Entropy

ObjectiveThe initial principal task of a Brain-Computer Interfacing (BCI) research is to extract the best feature set from a raw EEG (Electroencephalogram) signal so that it can be used for the classification of two or multiple different events. The main goal of the paper is to develop a comparative analysis among different feature extraction techniques and classification algorithms.Materials and methodsIn this present investigation, four different methodologies have been adopted to classify the recorded MI (motor imagery) EEG signal, and their comparative study has been reported. Haar Wavelet Energy (HWE), Band Power, Cross-correlation, and Spectral Entropy (SE) based Cross-correlation feature extraction techniques have been considered to obtain the necessary features set from the raw EEG signals. Four different machine learning algorithms, viz. LDA (Linear Discriminant Analysis), QDA (Quadratic Discriminant Analysis), Naïve Bayes, and Decision Tree, have been used to classify the features.ResultsThe best average classification accuracies are 92.50%, 93.12%, 72.26%, and 98.71% using the four methods. Further, these results have been compared with some recent existing methods.ConclusionThe comparative results indicate a significant accuracy level performance improvement of the proposed methods with respect to the existing one. Hence, this presented work can guide to select the best feature extraction method and the classifier algorithm for MI-based EEG signals.     

Improving the separability of multiple EEG features for a BCI by neural-time-series-prediction-preprocessing

A recently proposed method for EEG preprocessing is extended and analyzed in this work via a range of different tests in combination with various other BCI components. Neural-time-series-prediction-processing (NTSPP) is a predictive approach to EEG preprocessing where prediction models (PMs) are trained to perform one-step-ahead prediction of the EEG times-series which reflect motor imagery induced alterations in neuronal activity. Due to the specialization of distinct PMs, the predicted signals (Ys) and error signals (Es) are distinctly different from the original (Os) signals. The PMs map the Os signals to a higher dimension which, in the majority of cases, produces features that are more separable than those produced by the Os signals. Four feature extraction procedures, ranging in complexity and in terms of the information which is extracted i.e., time domain, frequency domain and time–frequency (t–f) domain, are used to determine the separability enhancements which are verified by comparative statistical tests and brain–computer interface (BCI) tests on six subjects. It is shown that, in the majority of the tests, features extracted from the NTSPP signals are more separable than those extracted from the Os signals, in terms of increased Euclidean distance between class means, reduced inter-class correlations and intra-class variance, and higher classification accuracy (CA), information transfer (IT) rate and mutual information (MI).     

A Generalizable Brain-Computer Interface (BCI) Using Machine Learning for Feature Discovery

This work describes a generalized method for classifying motor-related neural signals for a brain-computer interface (BCI), based on a stochastic machine learning method. The method differs from the various feature extraction and selection techniques employed in many other BCI systems. The classifier does not use extensive a-priori information, resulting in reduced reliance on highly specific domain knowledge. Instead of pre-defining features, the time-domain signal is input to a population of multi-layer perceptrons (MLPs) in order to perform a stochastic search for the best structure. The results showed that the average performance of the new algorithm outperformed other published methods using the Berlin BCI IV (2008) competition dataset and was comparable to the best results in the Berlin BCI II (2002–3) competition dataset. The new method was also applied to electroencephalography (EEG) data recorded from five subjects undertaking a hand squeeze task and demonstrated high levels of accuracy with a mean classification accuracy of 78.9% after five-fold cross-validation. Our new approach has been shown to give accurate results across different motor tasks and signal types as well as between subjects.     

Estimating the Intended Sound Direction of the User: Toward an Auditory Brain-Computer Interface Using Out-of-Head Sound Localization

The auditory Brain-Computer Interface (BCI) using electroencephalograms (EEG) is a subject of intensive study. As a cue, auditory BCIs can deal with many of the characteristics of stimuli such as tone, pitch, and voices. Spatial information on auditory stimuli also provides useful information for a BCI. However, in a portable system, virtual auditory stimuli have to be presented spatially through earphones or headphones, instead of loudspeakers. We investigated the possibility of an auditory BCI using the out-of-head sound localization technique, which enables us to present virtual auditory stimuli to users from any direction, through earphones. The feasibility of a BCI using this technique was evaluated in an EEG oddball experiment and offline analysis. A virtual auditory stimulus was presented to the subject from one of six directions. Using a support vector machine, we were able to classify whether the subject attended the direction of a presented stimulus from EEG signals. The mean accuracy across subjects was 70.0% in the single-trial classification. When we used trial-averaged EEG signals as inputs to the classifier, the mean accuracy across seven subjects reached 89.5% (for 10-trial averaging). Further analysis showed that the P300 event-related potential responses from 200 to 500 ms in central and posterior regions of the brain contributed to the classification. In comparison with the results obtained from a loudspeaker experiment, we confirmed that stimulus presentation by out-of-head sound localization achieved similar event-related potential responses and classification performances. These results suggest that out-of-head sound localization enables us to provide a high-performance and loudspeaker-less portable BCI system.     

Selective Sensation Based Brain-Computer Interface via Mechanical Vibrotactile Stimulation

In this work, mechanical vibrotactile stimulation was applied to subjects’ left and right wrist skins with equal intensity, and a selective sensation perception task was performed to achieve two types of selections similar to motor imagery Brain-Computer Interface. The proposed system was based on event-related desynchronization/synchronization (ERD/ERS), which had a correlation with processing of afferent inflow in human somatosensory system, and attentional effect which modulated the ERD/ERS. The experiments were carried out on nine subjects (without experience in selective sensation), and six of them showed a discrimination accuracy above 80%, three of them above 95%. Comparative experiments with motor imagery (with and without presence of stimulation) were also carried out, which further showed the feasibility of selective sensation as an alternative BCI task complementary to motor imagery. Specifically there was significant improvement () from near 65% in motor imagery (with and without presence of stimulation) to above 80% in selective sensation on some subjects. The proposed BCI modality might well cooperate with existing BCI modalities in the literature in enlarging the widespread usage of BCI system.