Estimating the Intended Sound Direction of the User: Toward an Auditory Brain-Computer Interface Using Out-of-Head Sound Localization

The auditory Brain-Computer Interface (BCI) using electroencephalograms (EEG) is a subject of intensive study. As a cue, auditory BCIs can deal with many of the characteristics of stimuli such as tone, pitch, and voices. Spatial information on auditory stimuli also provides useful information for a BCI. However, in a portable system, virtual auditory stimuli have to be presented spatially through earphones or headphones, instead of loudspeakers. We investigated the possibility of an auditory BCI using the out-of-head sound localization technique, which enables us to present virtual auditory stimuli to users from any direction, through earphones. The feasibility of a BCI using this technique was evaluated in an EEG oddball experiment and offline analysis. A virtual auditory stimulus was presented to the subject from one of six directions. Using a support vector machine, we were able to classify whether the subject attended the direction of a presented stimulus from EEG signals. The mean accuracy across subjects was 70.0% in the single-trial classification. When we used trial-averaged EEG signals as inputs to the classifier, the mean accuracy across seven subjects reached 89.5% (for 10-trial averaging). Further analysis showed that the P300 event-related potential responses from 200 to 500 ms in central and posterior regions of the brain contributed to the classification. In comparison with the results obtained from a loudspeaker experiment, we confirmed that stimulus presentation by out-of-head sound localization achieved similar event-related potential responses and classification performances. These results suggest that out-of-head sound localization enables us to provide a high-performance and loudspeaker-less portable BCI system.     

EEG-based communication and control: speed-accuracy relationships

People can learn to control mu (8–12 Hz) or beta (18–25 Hz) rhythm amplitude in the EEG recorded over sensorimotor cortex and use it to move a cursor to a target on a video screen. In our current EEG-based brain–computer interface (BCI) system, cursor movement is a linear function of mu or beta rhythm amplitude. In order to maximize the participant's control over the direction of cursor movement, the intercept in this equation is kept equal to the mean amplitude of recent performance. Selection of the optimal slope, or gain, which determines the magnitude of the individual cursor movements, is a more difficult problem. This study examined the relationship between gain and accuracy in a 1-dimensional EEG-based cursor movement task in which individuals select among 2 or more choices by holding the cursor at the desired choice for a fixed period of time (i.e., the dwell time). With 4 targets arranged in a vertical column on the screen, large gains favored the end targets whereas smaller gains favored the central targets. In addition, manipulating gain and dwell time within participants produces results that are in agreement with simulations based on a simple theoretical model of performance. Optimal performance occurs when correct selection of targets is uniform across position. Thus, it is desirable to remove any trend in the function relating accuracy to target position. We evaluated a controller that is designed to minimize the linear and quadratic trends in the accuracy with which participants hit the 4 targets. These results indicate that gain should be adjusted to the individual participants, and suggest that continual online gain adaptation could increase the speed and accuracy of EEG-based cursor control.     

Improving N1 classification by grouping EEG trials with phases of pre-stimulus EEG oscillations

A reactive brain-computer interface using electroencephalography (EEG) relies on the classification of evoked ERP responses. As the trial-to-trial variation is evitable in EEG signals, it is a challenge to capture the consistent classification features distribution. Clustering EEG trials with similar features and utilizing a specific classifier adjusted to each cluster can improve EEG classification. In this paper, instead of measuring the similarity of ERP features, the brain states during image stimuli presentation that evoked N1 responses were used to group EEG trials. The correlation between momentary phases of pre-stimulus EEG oscillations and N1 amplitudes was analyzed. The results demonstrated that the phases of time–frequency points about 5.3 Hz and 0.3 s before the stimulus onset have significant effect on the ERP classification accuracy. Our findings revealed that N1 components in ERP fluctuated with momentary phases of EEG. We also further studied the influence of pre-stimulus momentary phases on classification of N1 features. Results showed that linear classifiers demonstrated outstanding classification performance when training and testing trials have close momentary phases. Therefore, this gave us a new direction to improve EEG classification by grouping EEG trials with similar pre-stimulus phases and using each to train unit classifiers respectively.     

The Narcotrend Index: classification algorithm, correlation with propofol effect-site concentrations, and comparison with spectral parameters.

A reliable assessment of the depth of hypnosis during sedation and general anaesthesia using the EEG is a subject of current interest. The Narcotrend Index implemented in the latest version 4.0 of the EEG monitor Narcotrend provides an automatic classification of the EEG on a scale ranging from 100 (awake) to 0 (very deep hypnosis, EEG suppression). The classification algorithms implemented in the EEG monitor Narcotrend are described. In a study the correlation of the propofol effect-site concentration with the Narcotrend Index and with the traditional spectral parameters total power, relative power in the standard frequency bands delta, theta, alpha, and beta, median frequency, 95% spectral edge frequency, burst-compensated spectral edge frequency, and spectral entropy was investigated. The Narcotrend Index had the highest average correlation with the propofol effect-site concentration and the smallest variability of the individual correlation values. Moreover, the Narcotrend Index was the only parameter which showed a monophasic trend over the whole investigated time period. The Narcotrend monitor can make a significant contribution to the improvement of the quality of anaesthesia by adjusting the dosage of hypnotics to individual patient needs.     

Automatic classification algorithms of the EEG monitor Narcotrend for routinely recorded EEG data from general anaesthesia: a validation study.

Impacts of hypnotic drugs on brain function are reflected in the EEG. The EEG monitor Narcotrend performs an automatic classification of the EEG using a scale which was proposed by Kugler for visual evaluation of the EEG. In this article the results of a validation study of the automatic classification algorithms implemented in the EEG monitor Narcotrend are presented. Visual and automatic classification of EEG data recorded in routine clinical practice were compared. The correlation between visual and automatic assessment was high (Spearman rank correlation r = 0.90, prediction probability Pk = 0.90) and a sufficient agreement between visual and automatic assessment was achieved for 92% of the analysed EEG epochs. The results of the study suggest that the automatic classification algorithms implemented in the EEG monitor Narcotrend yield a reliable assessment of the depth of hypnosis.     

A Supplementary System for a Brain-Machine Interface Based on Jaw Artifacts for the Bidimensional Control of a Robotic Arm

Non-invasive Brain-Machine Interfaces (BMIs) are being used more and more these days to design systems focused on helping people with motor disabilities. Spontaneous BMIs translate user''s brain signals into commands to control devices. On these systems, by and large, 2 different mental tasks can be detected with enough accuracy. However, a large training time is required and the system needs to be adjusted on each session. This paper presents a supplementary system that employs BMI sensors, allowing the use of 2 systems (the BMI system and the supplementary system) with the same data acquisition device. This supplementary system is designed to control a robotic arm in two dimensions using electromyographical (EMG) signals extracted from the electroencephalographical (EEG) recordings. These signals are voluntarily produced by users clenching their jaws. EEG signals (with EMG contributions) were registered and analyzed to obtain the electrodes and the range of frequencies which provide the best classification results for 5 different clenching tasks. A training stage, based on the 2-dimensional control of a cursor, was designed and used by the volunteers to get used to this control. Afterwards, the control was extrapolated to a robotic arm in a 2-dimensional workspace. Although the training performed by volunteers requires 70 minutes, the final results suggest that in a shorter period of time (45 min), users should be able to control the robotic arm in 2 dimensions with their jaws. The designed system is compared with a similar 2-dimensional system based on spontaneous BMIs, and our system shows faster and more accurate performance. This is due to the nature of the control signals. Brain potentials are much more difficult to control than the electromyographical signals produced by jaw clenches. Additionally, the presented system also shows an improvement in the results compared with an electrooculographic system in a similar environment.     

Adapting subject specific motor imagery EEG patterns in space–time–frequency for a brain computer interface

In this paper we propose a new technique that adaptively extracts subject specific motor imagery related EEG patterns in the space–time–frequency plane for single trial classification. The proposed approach requires no prior knowledge of reactive frequency bands, their temporal behavior or cortical locations. For a given electrode array, it finds all these parameters by constructing electrode adaptive time–frequency segmentations that are optimized for discrimination. This is accomplished first by segmenting the EEG along the time axis with Local Cosine Packets. Next the most discriminant frequency subbands are selected in each time segment with a frequency axis clustering algorithm to achieve time and frequency band adaptation individually. Finally the subject adapted features are sorted according to their discrimination power to reduce dimensionality and the top subset is used for final classification. We provide experimental results for 5 subjects of the BCI competition 2005 dataset IVa to show the superior performance of the proposed method. In particular, we demonstrate that by using a linear support vector machine as a classifier, the classification accuracy of the proposed algorithm varied between 90.5% and 99.7% and the average classification accuracy was 96%.     

Potential of the learning vector quantizer in the cell classification of endometrial lesions in postmenopausal women

OBJECTIVE: To investigate the potential of artificial neural networks for cell identification in endometrial lesions from postmenopausal women. STUDY DESIGN: The study was performed on cytologic material obtained by the Gynoscann endometrial cell samplerfrom 12 cases of atrophic endometrium, 48 cases of hyperplasia without cytologic atypia (18 cases of simple hyperplasia and 30 cases of complex hyperplasia), 12 cases of hyperplasia with cytologic atypia (complex atypical hyperplasia) and 48 cases of adenocarcinoma (30 cases of well-differentiated, 12 cases of moderately differentiated and 6 cases of poorly differentiated carcinoma). From each case approximately 100 cells were examined using a custom image analysis system. A learning vector quantizer (LVQ) identified the collected data. RESULTS: Investigation of cells from Endometrial Alterations with LVQ proved that according to the nuclear characteristics, as expressed by morphometric and textural measures, the endometrial cells from postmenopausal women may be identified as belonging to one of thefollowing three groups: atrophy, hyperplasia without cytologic atypia (simple and complex hyperplasia) and malignant neoplastic lesions (atypical complex and adenocarcinoma). CONCLUSION: The role of nuclear morphologic features in the cytologic diagnosis of endometrial alterations was confirmed. The overlap in thefeature space observed indicates that cell characteristics do not form strictly separate clusters. Thatfact explains the difficulty that morphologists have with the reproducible identification of cells from endometrial lesions in postmenopausal women. Application of LVQ offers a good classification at the cell level and promises to be a powerful toolfor classification on the individual patient level andfor the clarification of the natural history of endometrial pathology.     

Feature selection for sleep/wake stages classification using data driven methods

This paper focuses on the problem of selecting relevant features extracted from human polysomnographic (PSG) signals to perform accurate sleep/wake stages classification. Extraction of various features from the electroencephalogram (EEG), the electro-oculogram (EOG) and the electromyogram (EMG) processed in the frequency and time domains was achieved using a database of 47 night sleep recordings obtained from healthy adults in laboratory settings. Multiple iterative feature selection and supervised classification methods were applied together with a systematic statistical assessment of the classification performances. Our results show that using a simple set of features such as relative EEG powers in five frequency bands yields an agreement of 71% with the whole database classification of two human experts. These performances are within the range of existing classification systems. The addition of features extracted from the EOG and EMG signals makes it possible to reach about 80% of agreement with the expert classification. The most significant improvement on classification accuracy is obtained on NREM sleep stage I, a stage of transition between sleep and wakefulness.     

Model analyses of visual biofeedback training for EEG-based brain-computer interface

The primary goal of this study was to construct a simulation model of a biofeedback brain-computer interface (BCI) system to analyze the effect of biofeedback training on BCI users. A mathematical model of a man-machine visual-biofeedback BCI system was constructed to simulate a subject using a BCI system to control cursor movements. The model consisted of a visual tracking system, a thalamo-cortical model for EEG generation, and a BCI system. The BCI system in the model was realized for real experiments of visual biofeedback training. Ten sessions of visual biofeedback training were performed in eight normal subjects during a 3-week period. The task was to move a cursor horizontally across a screen, or to hold it at the screen’s center. Experimental conditions and EEG data obtained from real experiments were then simulated with the model. Three model parameters, representing the adaptation rate of gain in the visual tracking system and the relative synaptic strength between the thalamic reticular and thalamo-cortical cells in the Rolandic areas, were estimated by optimization techniques so that the performance of the model best fitted the experimental results. The serial changes of these parameters over the ten sessions, reflecting the effects of biofeedback training, were analyzed. The model simulation could reproduce results similar to the experimental data. The group mean success rate and information transfer rate improved significantly after training (56.6 to 81.1% and 0.19 to 0.76 bits/trial, respectively). All three model parameters displayed similar and statistically significant increasing trends with time. Extensive simulation with systematic changes of these parameters also demonstrated that assigning larger values to the parameters improved the BCI performance. We constructed a model of a biofeedback BCI system that could simulate experimental data and the effect of training. The simulation results implied that the improvement was achieved through a quicker adaptation rate in visual tracking gain and a larger synaptic gain from the visual tracking system to the thalamic reticular cells. In addition to the purpose of this study, the constructed biofeedback BCI model can also be used both to investigate the effects of different biofeedback paradigms and to test, estimate, or predict the performances of other newly developed BCI signal processing algorithms.     

Research on the relation of EEG signal chaos characteristics with high-level intelligence activity of human brain

Using phase space reconstruct technique from one-dimensional and multi-dimensional time series and the quantitative criterion rule of system chaos, and combining the neural network; analyses, computations and sort are conducted on electroencephalogram (EEG) signals of five kinds of human consciousness activities (relaxation, mental arithmetic of multiplication, mental composition of a letter, visualizing a 3-dimensional object being revolved about an axis, and visualizing numbers being written or erased on a blackboard). Through comparative studies on the determinacy, the phase graph, the power spectra, the approximate entropy, the correlation dimension and the Lyapunov exponent of EEG signals of 5 kinds of consciousness activities, the following conclusions are shown: (1) The statistic results of the deterministic computation indicate that chaos characteristic may lie in human consciousness activities, and central tendency measure (CTM) is consistent with phase graph, so it can be used as a division way of EEG attractor. (2) The analyses of power spectra show that ideology of single subject is almost identical but the frequency channels of different consciousness activities have slight difference. (3) The approximate entropy between different subjects exist discrepancy. Under the same conditions, the larger the approximate entropy of subject is, the better the subject's innovation is. (4) The results of the correlation dimension and the Lyapunov exponent indicate that activities of human brain exist in attractors with fractional dimensions. (5) Nonlinear quantitative criterion rule, which unites the neural network, can classify different kinds of consciousness activities well. In this paper, the results of classification indicate that the consciousness activity of arithmetic has better differentiation degree than that of abstract.     

The adaptive classification of the dynamic spectral patterns in the human EEG]

The classification technique of single spectra based on a matrix of intercorrelation between these spectra and the fixed set of standard spectral patterns (SP) has been put forward. Including in the classification technique a special procedure for automatic adaptation of the standard SP to given EEG records makes it possible to reduce the number of unclassified single spectra to a minimum (6-10%), which we can ignore during comparative analysis of the EEG classification profiles. Using the universal set of standard SP makes it possible to compare the results of classification of different EEG records. The results of the analysis of classification profiles of human multichannel EEG during performance of the memory task on perception of visual images are described in the paper. It has been shown that both the total EEG reorganization associated with the alpha rhythm blockade during eyes opening and less noticeable EEG shifts accompanying changes in the stages of cognitive activity are underlain by a rather differentiated transformations of relative contributions of each type of the SP into the total power spectrum. It has been revealed that a relatively small part (15-20%) of the elementary EEG segments participates in the reorganization of the EEG classification profile.     

Brain-computer interface based on generation of visual images

This paper examines the task of recognizing EEG patterns that correspond to performing three mental tasks: relaxation and imagining of two types of pictures: faces and houses. The experiments were performed using two EEG headsets: BrainProducts ActiCap and Emotiv EPOC. The Emotiv headset becomes widely used in consumer BCI application allowing for conducting large-scale EEG experiments in the future. Since classification accuracy significantly exceeded the level of random classification during the first three days of the experiment with EPOC headset, a control experiment was performed on the fourth day using ActiCap. The control experiment has shown that utilization of high-quality research equipment can enhance classification accuracy (up to 68% in some subjects) and that the accuracy is independent of the presence of EEG artifacts related to blinking and eye movement. This study also shows that computationally-inexpensive bayesian classifier based on covariance matrix analysis yields similar classification accuracy in this problem as a more sophisticated Multi-class Common Spatial Patterns (MCSP) classifier.     

基于小波包熵的运动意识任务分类研究

提出了以小波包熵作为脑电特征向量的左右手运动意识任务分类方法,对被测试者想象左右手运动时的脑电小波包熵动态变化情况及分析窗口长度的选择进行了研究.结果表明,小波包熵能很好地反映左右手运动想象的脑电特征变化,用线性判别式算法对脑电特征进行识别,分类正确率达到92.14%.由于小波包熵的计算比较简单,稳定性好,识别率高,为大脑运动意识任务的分类提供了新思路.     

Classification of Motor Imagery EEG Based on Time-Domain and Frequency-Domain Dual-Stream Convolutional Neural Network

Background and objectiveAn important task of the brain-computer interface (BCI) of motor imagery is to extract effective time-domain features, frequency-domain features or time-frequency domain features from the raw electroencephalogram (EEG) signals for classification of motor imagery. However, choosing an appropriate method to combine time domain and frequency domain features to improve the performance of motor imagery recognition is still a research hotspot.MethodsIn order to fully extract and utilize the time-domain and frequency-domain features of EEG in classification tasks, this paper proposed a novel dual-stream convolutional neural network (DCNN), which can use time domain signal and frequency domain signal as the inputs, and the extracted time-domain features and frequency-domain features are fused by linear weighting for classification training. Furthermore, the weight can be learned by the DCNN automatically.ResultsThe experiments based on BCI competition II dataset III and BCI competition IV dataset 2a showed that the model proposed by this study has better performance than other conventional methods. The model used time-frequency signal as the inputs had better performance than the model only used time-domain signals or frequency-domain signals. The accuracy of classification was improved for each subject compared with the models only used one signals as the inputs.ConclusionsFurther analysis shown that the fusion weight of different subject is specifically, adjusting the weight coefficient automatically is helpful to improve the classification accuracy.     

Individually Adapted Imagery Improves Brain-Computer Interface Performance in End-Users with Disability

Brain-computer interfaces (BCIs) translate oscillatory electroencephalogram (EEG) patterns into action. Different mental activities modulate spontaneous EEG rhythms in various ways. Non-stationarity and inherent variability of EEG signals, however, make reliable recognition of modulated EEG patterns challenging. Able-bodied individuals who use a BCI for the first time achieve - on average - binary classification performance of about 75%. Performance in users with central nervous system (CNS) tissue damage is typically lower. User training generally enhances reliability of EEG pattern generation and thus also robustness of pattern recognition. In this study, we investigated the impact of mental tasks on binary classification performance in BCI users with central nervous system (CNS) tissue damage such as persons with stroke or spinal cord injury (SCI). Motor imagery (MI), that is the kinesthetic imagination of movement (e.g. squeezing a rubber ball with the right hand), is the "gold standard" and mainly used to modulate EEG patterns. Based on our recent results in able-bodied users, we hypothesized that pair-wise combination of "brain-teaser" (e.g. mental subtraction and mental word association) and "dynamic imagery" (e.g. hand and feet MI) tasks significantly increases classification performance of induced EEG patterns in the selected end-user group. Within-day (How stable is the classification within a day?) and between-day (How well does a model trained on day one perform on unseen data of day two?) analysis of variability of mental task pair classification in nine individuals confirmed the hypothesis. We found that the use of the classical MI task pair hand vs. feed leads to significantly lower classification accuracy - in average up to 15% less - in most users with stroke or SCI. User-specific selection of task pairs was again essential to enhance performance. We expect that the gained evidence will significantly contribute to make imagery-based BCI technology become accessible to a larger population of users including individuals with special needs due to CNS damage.     

Online Classification of Multiple Motor Imagery Tasks Using Filter Bank Based Maximum-a-Posteriori Common Spatial Pattern Filters

ObjectiveThe main objective of this paper is to propose a novel technique, called filter bank maximum a-posteriori common spatial pattern (FB-MAP-CSP) algorithm, for online classification of multiple motor imagery activities using electroencephalography (EEG) signals. The proposed technique addresses the overfitting issue of CSP in addition to utilizing the spectral information of EEG signals inside the framework of filter banks while extending it to more than two conditions.Materials and methodsThe classification of motor imagery signals is based upon the detection of event-related de-synchronization (ERD) phenomena in the μ and β rhythms of EEG signals. Accordingly, two modifications in the existing MAP-CSP technique are presented: (i) The (pre-processed) EEG signals are spectrally filtered by a bank of filters lying in the μ and β brainwave frequency range, (ii) the framework of MAP-CSP is extended to deal with multiple (more than two) motor imagery tasks classification and the spatial filters thus obtained are calculated for each sub-band, separately. Subsequently, the most imperative features over all sub-bands are selected and un-regularized linear discriminant analysis is employed for classification of multiple motor imagery tasks.ResultsPublicly available dataset (BCI Competition IV Dataset I) is used to validate the proposed method i.e. FB-MAP-CSP. The results show that the proposed method yields superior classification results, in addition to be computationally more efficient in the case of online implementation, as compared to the conventional CSP based techniques and its variants for multiclass motor imagery classification.ConclusionThe proposed FB-MAP-CSP algorithm is found to be a potential / superior method for classifying multi-condition motor imagery EEG signals in comparison to FBCSP based techniques.     

Classification methods for ongoing EEG and MEG signals

Classification algorithms help predict the qualitative properties of a subject's mental state by extracting useful information from the highly multivariate non-invasive recordings of his brain activity. In particular, applying them to Magneto-encephalography (MEG) and electro-encephalography (EEG) is a challenging and promising task with prominent practical applications to e.g. Brain Computer Interface (BCI). In this paper, we first review the principles of the major classification techniques and discuss their application to MEG and EEG data classification. Next, we investigate the behavior of classification methods using real data recorded during a MEG visuomotor experiment. In particular, we study the influence of the classification algorithm, of the quantitative functional variables used in this classifier, and of the validation method. In addition, our findings suggest that by investigating the distribution of classifier coefficients, it is possible to infer knowledge and construct functional interpretations of the underlying neural mechanisms of the performed tasks. Finally, the promising results reported here (up to 97% classification accuracy on 1-second time windows) reflect the considerable potential of MEG for the continuous classification of mental states.     

Decision Tree in Working Memory Task Effectively Characterizes EEG Signals in Healthy Aging Adults

BackgroundThe changes in electroencephalogram (EEG) signals that reflect the changes in physiological structure, cognitive functions, and activities have been observed in healthy aging adults. It is unknown that when the brain aging initiates and whether these age-related alterations can be associated with incipient neurodegenerative diseases in healthy elderly individuals.Materials and methodsWe employed feature extraction and classification methods to classify and compare the EEG signals of middle-aged and elderly age groups. This study included 20 healthy middle-aged and 20 healthy elderly subjects. The EEG signals were recorded during a resting state (eyes-open and eyes-closed) and during a working memory (WM) task using eight electrodes. The minimum redundancy maximum relevance technique was employed in the selection of the optimal feature. Four classification methods, including decision tree, support vector machine, Naïve Bayes, and K-nearest neighbor, were used to distinguish the elderly age group from the middle-aged group based on their EEG signals.ResultsIn the resting state, a good correlation was observed among absolute power delta and theta bands and aging, whereas between beta absolute power and aging, a WM task correlation was observed. The results also indicated that the mean frequency and absolute power might be useful for the prediction and classification of EEG signals in aging individuals. Furthermore, the use of the decision tree method in a WM task state distinguished the elderly group from the middle-aged group with an accuracy of 87.5%.ConclusionsWorking memory could play a vital role in the optimization of classification of EEG signals in aging and discrimination of age-related issues associated with neurodegeneration.     

Nonnegative tensor factorization for continuous EEG classification

In this paper we present a method for continuous EEG classification, where we employ nonnegative tensor factorization (NTF) to determine discriminative spectral features and use the Viterbi algorithm to continuously classify multiple mental tasks. This is an extension of our previous work on the use of nonnegative matrix factorization (NMF) for EEG classification. Numerical experiments with two data sets in BCI competition, confirm the useful behavior of the method for continuous EEG classification.