An Efficient ERP-Based Brain-Computer Interface Using Random Set Presentation and Face Familiarity

Event-related potential (ERP)-based P300 spellers are commonly used in the field of brain-computer interfaces as an alternative channel of communication for people with severe neuro-muscular diseases. This study introduces a novel P300 based brain-computer interface (BCI) stimulus paradigm using a random set presentation pattern and exploiting the effects of face familiarity. The effect of face familiarity is widely studied in the cognitive neurosciences and has recently been addressed for the purpose of BCI. In this study we compare P300-based BCI performances of a conventional row-column (RC)-based paradigm with our approach that combines a random set presentation paradigm with (non-) self-face stimuli. Our experimental results indicate stronger deflections of the ERPs in response to face stimuli, which are further enhanced when using the self-face images, and thereby improving P300-based spelling performance. This lead to a significant reduction of stimulus sequences required for correct character classification. These findings demonstrate a promising new approach for improving the speed and thus fluency of BCI-enhanced communication with the widely used P300-based BCI setup.     

Regularized common spatial patterns with subject-to-subject transfer of EEG signals

In the context of brain-computer interface (BCI) system, the common spatial patterns (CSP) method has been used to extract discriminative spatial filters for the classification of electroencephalogram (EEG) signals. However, the classification performance of CSP typically deteriorates when a few training samples are collected from a new BCI user. In this paper, we propose an approach that maintains or improves the recognition accuracy of the system with only a small size of training data set. The proposed approach is formulated by regularizing the classical CSP technique with the strategy of transfer learning. Specifically, we incorporate into the CSP analysis inter-subject information involving the same task, by minimizing the difference between the inter-subject features. Experimental results on two data sets from BCI competitions show that the proposed approach greatly improves the classification performance over that of the conventional CSP method; the transformed variant proved to be successful in almost every case, based on a small number of available training samples.     

The P300 based brain-computer interface: effect of stimulus position in a stimulus train

The P300 brain-computer interface (BCI) is currently the most efficient BCI. This interface is based on detection of the P300 wave of the brain potentials evoked when a symbol related to the intended input is highlighted. To increase operation speed of the P300 BCI, reduction of the number of stimuli repetitions is needed. This reduction leads to increase of the relative contribution to the input symbol detection from the reaction to the first target stimulus. It is known that the event-related potentials (ERP) to the first stimulus presentations can be different from the ERP to stimuli presented latter. In particular, the amplitude of responses to the first stimulus presentations is often increased, which is beneficial for their recognition by the BCI. However, this effect was not studied within the BCI framework. The current study examined the ERP obtained from healthy participants (n = 14) in the standard P300 BCI paradigm using 10 trials, as well as in the modified P300 BCI with stimuli presented on moving objects in triple-trial (n = 6) and single-trial (n = 6) stimulation modes. Increased ERP amplitude was observed in response to the first target stimuli in both conditions, as well as in the single-trial mode comparing to triple-trial. We discuss the prospects of using the specific features of the ERP to first stimuli and the single-trial ERP for optimizing the high-speed modes in the P300 BCIs.     

Channel Selection Based on Phase Measurement in P300-Based Brain-Computer Interface

Most EEG-based brain-computer interface (BCI) paradigms include specific electrode positions. As the structures and activities of the brain vary with each individual, contributing channels should be chosen based on original records of BCIs. Phase measurement is an important approach in EEG analyses, but seldom used for channel selections. In this paper, the phase locking and concentrating value-based recursive feature elimination approach (PLCV-RFE) is proposed to produce robust-EEG channel selections in a P300 speller. The PLCV-RFE, deriving from the phase resetting mechanism, measures the phase relation between EEGs and ranks channels by the recursive strategy. Data recorded from 32 electrodes on 9 subjects are used to evaluate the proposed method. The results show that the PLCV-RFE substantially reduces channel sets and improves recognition accuracies significantly. Moreover, compared with other state-of-the-art feature selection methods (SSNRSF and SVM-RFE), the PLCV-RFE achieves better performance. Thus the phase measurement is available in the channel selection of BCI and it may be an evidence to indirectly support that phase resetting is at least one reason for ERP generations.     

Comparison of Sensor Selection Mechanisms for an ERP-Based Brain-Computer Interface

A major barrier for a broad applicability of brain-computer interfaces (BCIs) based on electroencephalography (EEG) is the large number of EEG sensor electrodes typically used. The necessity for this results from the fact that the relevant information for the BCI is often spread over the scalp in complex patterns that differ depending on subjects and application scenarios. Recently, a number of methods have been proposed to determine an individual optimal sensor selection. These methods have, however, rarely been compared against each other or against any type of baseline. In this paper, we review several selection approaches and propose one additional selection criterion based on the evaluation of the performance of a BCI system using a reduced set of sensors. We evaluate the methods in the context of a passive BCI system that is designed to detect a P300 event-related potential and compare the performance of the methods against randomly generated sensor constellations. For a realistic estimation of the reduced system''s performance we transfer sensor constellations found on one experimental session to a different session for evaluation. We identified notable (and unanticipated) differences among the methods and could demonstrate that the best method in our setup is able to reduce the required number of sensors considerably. Though our application focuses on EEG data, all presented algorithms and evaluation schemes can be transferred to any binary classification task on sensor arrays.     

The Changing Face of P300 BCIs: A Comparison of Stimulus Changes in a P300 BCI Involving Faces,Emotion, and Movement

Background

One of the most common types of brain-computer interfaces (BCIs) is called a P300 BCI, since it relies on the P300 and other event-related potentials (ERPs). In the canonical P300 BCI approach, items on a monitor flash briefly to elicit the necessary ERPs. Very recent work has shown that this approach may yield lower performance than alternate paradigms in which the items do not flash but instead change in other ways, such as moving, changing colour or changing to characters overlaid with faces.

Methodology/Principal Findings

The present study sought to extend this research direction by parametrically comparing different ways to change items in a P300 BCI. Healthy subjects used a P300 BCI across six different conditions. Three conditions were similar to our prior work, providing the first direct comparison of characters flashing, moving, and changing to faces. Three new conditions also explored facial motion and emotional expression. The six conditions were compared across objective measures such as classification accuracy and bit rate as well as subjective measures such as perceived difficulty. In line with recent studies, our results indicated that the character flash condition resulted in the lowest accuracy and bit rate. All four face conditions (mean accuracy >91%) yielded significantly better performance than the flash condition (mean accuracy = 75%).

Conclusions/Significance

Objective results reaffirmed that the face paradigm is superior to the canonical flash approach that has dominated P300 BCIs for over 20 years. The subjective reports indicated that the conditions that yielded better performance were not considered especially burdensome. Therefore, although further work is needed to identify which face paradigm is best, it is clear that the canonical flash approach should be replaced with a face paradigm when aiming at increasing bit rate. However, the face paradigm has to be further explored with practical applications particularly with locked-in patients.     

A Deep Learning Architecture for P300 Detection with Brain-Computer Interface Application

In this paper, a brain-computer interface (BCI) system for character recognition is proposed based on the P300 signal. A P300 speller is used to spell the word or character without any muscle movement. P300 detection is the first step to detect the character from the electroencephalogram (EEG) signal. The character is recognized from the detected P300 signal. In this paper, sparse autoencoder (SAE) and stacked sparse autoencoder (SSAE) based feature extraction methods are proposed for P300 detection. This work also proposes a fusion of deep-features with the temporal features for P300 detection. A SSAE technique extracts high-level information about input data. The combination of SSAE features with the temporal features provides abstract and temporal information about the signal. An ensemble of weighted artificial neural network (EWANN) is proposed for P300 detection to minimize the variation among different classifiers. To provide more importance to the good classifier for final classification, a higher weightage is assigned to the better performing classifier. These weights are calculated from the cross-validation test. The model is tested on two different publicly available datasets, and the proposed method provides better or comparable character recognition performance than the state-of-the-art methods.     

A P300-based Brain-Computer Interface with Stimuli on Moving Objects: Four-Session Single-Trial and Triple-Trial Tests with a Game-Like Task Design

Brain-computer interfaces (BCIs) are tools for controlling computers and other devices without using muscular activity, employing user-controlled variations in signals recorded from the user’s brain. One of the most efficient noninvasive BCIs is based on the P300 wave of the brain’s response to stimuli and is therefore referred to as the P300 BCI. Many modifications of this BCI have been proposed to further improve the BCI’s characteristics or to better adapt the BCI to various applications. However, in the original P300 BCI and in all of its modifications, the spatial positions of stimuli were fixed relative to each other, which can impose constraints on designing applications controlled by this BCI. We designed and tested a P300 BCI with stimuli presented on objects that were freely moving on a screen at a speed of 5.4°/s. Healthy participants practiced a game-like task with this BCI in either single-trial or triple-trial mode within four sessions. At each step, the participants were required to select one of nine moving objects. The mean online accuracy of BCI-based selection was 81% in the triple-trial mode and 65% in the single-trial mode. A relatively high P300 amplitude was observed in response to targets in most participants. Self-rated interest in the task was high and stable over the four sessions (the medians in the 1st/4th sessions were 79/84% and 76/71% in the groups practicing in the single-trial and triple-trial modes, respectively). We conclude that the movement of stimulus positions relative to each other may not prevent the efficient use of the P300 BCI by people controlling their gaze, e.g., in robotic devices and in video games.     

LASSO based stimulus frequency recognition model for SSVEP BCIs

Steady-state visual evoked potential (SSVEP) has been increasingly used for the study of brain–computer interface (BCI). How to recognize SSVEP with shorter time and lower error rate is one of the key points to develop a more efficient SSVEP-based BCI. To achieve this goal, we make use of the sparsity constraint of the least absolute shrinkage and selection operator (LASSO) for the extraction of more discriminative features of SSVEP, and then we propose a LASSO model using the linear regression between electroencephalogram (EEG) recordings and the standard square-wave signals of different frequencies to recognize SSVEP without the training stage. In this study, we verified the proposed LASSO model offline with the EEG data of nine healthy subjects in contrast to canonical correlation analysis (CCA). In the experiment, when a shorter time window was used, we found that the LASSO model yielded better performance in extracting robust and detectable features of SSVEP, and the information transfer rate obtained by the LASSO model was significantly higher than that of the CCA. Our proposed method can assist to reduce the recording time without sacrificing the classification accuracy and is promising for a high-speed SSVEP-based BCI.     

A New Auditory Multi-Class Brain-Computer Interface Paradigm: Spatial Hearing as an Informative Cue

Most P300-based brain-computer interface (BCI) approaches use the visual modality for stimulation. For use with patients suffering from amyotrophic lateral sclerosis (ALS) this might not be the preferable choice because of sight deterioration. Moreover, using a modality different from the visual one minimizes interference with possible visual feedback. Therefore, a multi-class BCI paradigm is proposed that uses spatially distributed, auditory cues. Ten healthy subjects participated in an offline oddball task with the spatial location of the stimuli being a discriminating cue. Experiments were done in free field, with an individual speaker for each location. Different inter-stimulus intervals of 1000 ms, 300 ms and 175 ms were tested. With averaging over multiple repetitions, selection scores went over 90% for most conditions, i.e., in over 90% of the trials the correct location was selected. One subject reached a 100% correct score. Corresponding information transfer rates were high, up to an average score of 17.39 bits/minute for the 175 ms condition (best subject 25.20 bits/minute). When presenting the stimuli through a single speaker, thus effectively canceling the spatial properties of the cue, selection scores went down below 70% for most subjects. We conclude that the proposed spatial auditory paradigm is successful for healthy subjects and shows promising results that may lead to a fast BCI that solely relies on the auditory sense.     

Exploring Combinations of Auditory and Visual Stimuli for Gaze-Independent Brain-Computer Interfaces

For Brain-Computer Interface (BCI) systems that are designed for users with severe impairments of the oculomotor system, an appropriate mode of presenting stimuli to the user is crucial. To investigate whether multi-sensory integration can be exploited in the gaze-independent event-related potentials (ERP) speller and to enhance BCI performance, we designed a visual-auditory speller. We investigate the possibility to enhance stimulus presentation by combining visual and auditory stimuli within gaze-independent spellers. In this study with N = 15 healthy users, two different ways of combining the two sensory modalities are proposed: simultaneous redundant streams (Combined-Speller) and interleaved independent streams (Parallel-Speller). Unimodal stimuli were applied as control conditions. The workload, ERP components, classification accuracy and resulting spelling speed were analyzed for each condition. The Combined-speller showed a lower workload than uni-modal paradigms, without the sacrifice of spelling performance. Besides, shorter latencies, lower amplitudes, as well as a shift of the temporal and spatial distribution of discriminative information were observed for Combined-speller. These results are important and are inspirations for future studies to search the reason for these differences. For the more innovative and demanding Parallel-Speller, where the auditory and visual domains are independent from each other, a proof of concept was obtained: fifteen users could spell online with a mean accuracy of 87.7% (chance level <3%) showing a competitive average speed of 1.65 symbols per minute. The fact that it requires only one selection period per symbol makes it a good candidate for a fast communication channel. It brings a new insight into the true multisensory stimuli paradigms. Novel approaches for combining two sensory modalities were designed here, which are valuable for the development of ERP-based BCI paradigms.     

<Emphasis Type="Italic">P</Emphasis>300-based brain-computer interface: The effect of the stimulus position in a stimulus train

The most popular type of brain-computer interfaces (BCIs) are based on the detection of the P300 wave of the evoked potentials appearing in response to a stimulus chosen by the subject. In order to increase the speed of operation of these BCIs, it is possible to decrease the number of repeated stimulus presentations. It is associated with an increase in the relative importance of the response to the first stimulus in a train for correct recognition of the stimulus chosen. Event-related potentials (ERPs) in response to the first stimulus presentations are known to have their own specificity. Particularly, in many cases, the amplitude of the response to the first presentations is enhanced, which makes it very suitable for recognition in a BCI. However, this effect has not been studied to date. In this study, the ERPs recorded in healthy subjects in a standard BCI paradigm (n = 14) with ten presentations of stimuli or during triple-trial (n = 6) and single-trial (n = 6) presentations of stimuli in a modified BCI paradigm with moving objects have been analyzed. In both cases, first presentations of the target stimuli or single-trial presentation of the target stimulus were associated with higher amplitudes of ERPs. The opportunity to use specific differences between the responses to the first or single-trial presentations and the responses to later stimuli during their repeated presentations for improving high-speed operations in the P300-based BCI is discussed.     

Semi-supervised joint spatio-temporal feature selection for P300-based BCI speller

In this paper, we address the important problem of feature selection for a P300-based brain computer interface (BCI) speller system in several aspects. Firstly, time segment selection and electroencephalogram channel selection are jointly performed for better discriminability of P300 and background signals. Secondly, in view of the situation that training data with labels are insufficient, we propose an iterative semi-supervised support vector machine for joint spatio-temporal feature selection as well as classification, in which both labeled training data and unlabeled test data are utilized. More importantly, the semi-supervised learning enables the adaptivity of the system. The performance of our algorithm has been evaluated through the analysis of a P300 dataset provided by BCI Competition 2005 and another dataset collected from an in-house P300 speller system. The results show that our algorithm for joint feature selection and classification achieves satisfactory performance, meanwhile it can significantly reduce the training effort of the system. Furthermore, this algorithm is implemented online and the corresponding results demonstrate that our algorithm can improve the adaptiveness of the P300-based BCI speller.     

A Generalizable Brain-Computer Interface (BCI) Using Machine Learning for Feature Discovery

This work describes a generalized method for classifying motor-related neural signals for a brain-computer interface (BCI), based on a stochastic machine learning method. The method differs from the various feature extraction and selection techniques employed in many other BCI systems. The classifier does not use extensive a-priori information, resulting in reduced reliance on highly specific domain knowledge. Instead of pre-defining features, the time-domain signal is input to a population of multi-layer perceptrons (MLPs) in order to perform a stochastic search for the best structure. The results showed that the average performance of the new algorithm outperformed other published methods using the Berlin BCI IV (2008) competition dataset and was comparable to the best results in the Berlin BCI II (2002–3) competition dataset. The new method was also applied to electroencephalography (EEG) data recorded from five subjects undertaking a hand squeeze task and demonstrated high levels of accuracy with a mean classification accuracy of 78.9% after five-fold cross-validation. Our new approach has been shown to give accurate results across different motor tasks and signal types as well as between subjects.     

Classification of Motor Imagery EEG Based on Time-Domain and Frequency-Domain Dual-Stream Convolutional Neural Network

Background and objectiveAn important task of the brain-computer interface (BCI) of motor imagery is to extract effective time-domain features, frequency-domain features or time-frequency domain features from the raw electroencephalogram (EEG) signals for classification of motor imagery. However, choosing an appropriate method to combine time domain and frequency domain features to improve the performance of motor imagery recognition is still a research hotspot.MethodsIn order to fully extract and utilize the time-domain and frequency-domain features of EEG in classification tasks, this paper proposed a novel dual-stream convolutional neural network (DCNN), which can use time domain signal and frequency domain signal as the inputs, and the extracted time-domain features and frequency-domain features are fused by linear weighting for classification training. Furthermore, the weight can be learned by the DCNN automatically.ResultsThe experiments based on BCI competition II dataset III and BCI competition IV dataset 2a showed that the model proposed by this study has better performance than other conventional methods. The model used time-frequency signal as the inputs had better performance than the model only used time-domain signals or frequency-domain signals. The accuracy of classification was improved for each subject compared with the models only used one signals as the inputs.ConclusionsFurther analysis shown that the fusion weight of different subject is specifically, adjusting the weight coefficient automatically is helpful to improve the classification accuracy.     

Design of an EEG-based brain-computer interface (BCI) from standard components running in real-time under Windows.

An EEG-based brain-computer interface (BCI) is a direct connection between the human brain and the computer. Such a communication system is needed by patients with severe motor impairments (e.g. late stage of Amyotrophic Lateral Sclerosis) and has to operate in real-time. This paper describes the selection of the appropriate components to construct such a BCI and focuses also on the selection of a suitable programming language and operating system. The multichannel system runs under Windows 95, equipped with a real-time Kernel expansion to obtain reasonable real-time operations on a standard PC. Matlab controls the data acquisition and the presentation of the experimental paradigm, while Simulink is used to calculate the recursive least square (RLS) algorithm that describes the current state of the EEG in real-time. First results of the new low-cost BCI show that the accuracy of differentiating imagination of left and right hand movement is around 95%.     

Overlapped Partitioning for Ensemble Classifiers of P300-Based Brain-Computer Interfaces

A P300-based brain-computer interface (BCI) enables a wide range of people to control devices that improve their quality of life. Ensemble classifiers with naive partitioning were recently applied to the P300-based BCI and these classification performances were assessed. However, they were usually trained on a large amount of training data (e.g., 15300). In this study, we evaluated ensemble linear discriminant analysis (LDA) classifiers with a newly proposed overlapped partitioning method using 900 training data. In addition, the classification performances of the ensemble classifier with naive partitioning and a single LDA classifier were compared. One of three conditions for dimension reduction was applied: the stepwise method, principal component analysis (PCA), or none. The results show that an ensemble stepwise LDA (SWLDA) classifier with overlapped partitioning achieved a better performance than the commonly used single SWLDA classifier and an ensemble SWLDA classifier with naive partitioning. This result implies that the performance of the SWLDA is improved by overlapped partitioning and the ensemble classifier with overlapped partitioning requires less training data than that with naive partitioning. This study contributes towards reducing the required amount of training data and achieving better classification performance.     

Estimating the Intended Sound Direction of the User: Toward an Auditory Brain-Computer Interface Using Out-of-Head Sound Localization

The auditory Brain-Computer Interface (BCI) using electroencephalograms (EEG) is a subject of intensive study. As a cue, auditory BCIs can deal with many of the characteristics of stimuli such as tone, pitch, and voices. Spatial information on auditory stimuli also provides useful information for a BCI. However, in a portable system, virtual auditory stimuli have to be presented spatially through earphones or headphones, instead of loudspeakers. We investigated the possibility of an auditory BCI using the out-of-head sound localization technique, which enables us to present virtual auditory stimuli to users from any direction, through earphones. The feasibility of a BCI using this technique was evaluated in an EEG oddball experiment and offline analysis. A virtual auditory stimulus was presented to the subject from one of six directions. Using a support vector machine, we were able to classify whether the subject attended the direction of a presented stimulus from EEG signals. The mean accuracy across subjects was 70.0% in the single-trial classification. When we used trial-averaged EEG signals as inputs to the classifier, the mean accuracy across seven subjects reached 89.5% (for 10-trial averaging). Further analysis showed that the P300 event-related potential responses from 200 to 500 ms in central and posterior regions of the brain contributed to the classification. In comparison with the results obtained from a loudspeaker experiment, we confirmed that stimulus presentation by out-of-head sound localization achieved similar event-related potential responses and classification performances. These results suggest that out-of-head sound localization enables us to provide a high-performance and loudspeaker-less portable BCI system.     

An exploration of spatial auditory BCI paradigms with different sounds: music notes versus beeps

Visual brain-computer interfaces (BCIs) are not suitable for people who cannot reliably maintain their eye gaze. Considering that this group usually maintains audition, an auditory based BCI may be a good choice for them. In this paper, we explore two auditory patterns: (1) a pattern utilizing symmetrical spatial cues with multiple frequency beeps [called the high low medium (HLM) pattern], and (2) a pattern utilizing non-symmetrical spatial cues with six tones derived from the diatonic scale [called the diatonic scale (DS) pattern]. These two patterns are compared to each other in terms of accuracy to determine which auditory pattern is better. The HLM pattern uses three different frequency beeps and has a symmetrical spatial distribution. The DS pattern uses six spoken stimuli, which are six notes solmizated as “do”, “re”, “mi”, “fa”, “sol” and “la”, and derived from the diatonic scale. These six sounds are distributed to six, spatially distributed, speakers. Thus, we compare a BCI paradigm using beeps with another BCI paradigm using tones on the diatonic scale, when the stimuli are spatially distributed. Although no significant differences are found between the ERPs, the HLM pattern performs better than the DS pattern: the online accuracy achieved with the HLM pattern is significantly higher than that achieved with the DS pattern (p = 0.0028).     

A Novel Algorithm to Enhance P300 in Single Trials: Application to Lie Detection Using F-Score and SVM

The investigation of lie detection methods based on P300 potentials has drawn much interest in recent years. We presented a novel algorithm to enhance signal-to-noise ratio (SNR) of P300 and applied it in lie detection to increase the classification accuracy. Thirty-four subjects were divided randomly into guilty and innocent groups, and the EEG signals on 14 electrodes were recorded. A novel spatial denoising algorithm (SDA) was proposed to reconstruct the P300 with a high SNR based on independent component analysis. The differences between the proposed method and our/other early published methods mainly lie in the extraction and feature selection method of P300. Three groups of features were extracted from the denoised waves; then, the optimal features were selected by the F-score method. Selected feature samples were finally fed into three classical classifiers to make a performance comparison. The optimal parameter values in the SDA and the classifiers were tuned using a grid-searching training procedure with cross-validation. The support vector machine (SVM) approach was adopted to combine with an F-score because this approach had the best performance. The presented model F-score_SVM reaches a significantly higher classification accuracy for P300 (specificity of 96.05%) and non-P300 (sensitivity of 96.11%) compared with the results obtained without using SDA and compared with the results obtained by other classification models. Moreover, a higher individual diagnosis rate can be obtained compared with previous methods, and the presented method requires only a small number of stimuli in the real testing application.