Semi-supervised joint spatio-temporal feature selection for P300-based BCI speller

In this paper, we address the important problem of feature selection for a P300-based brain computer interface (BCI) speller system in several aspects. Firstly, time segment selection and electroencephalogram channel selection are jointly performed for better discriminability of P300 and background signals. Secondly, in view of the situation that training data with labels are insufficient, we propose an iterative semi-supervised support vector machine for joint spatio-temporal feature selection as well as classification, in which both labeled training data and unlabeled test data are utilized. More importantly, the semi-supervised learning enables the adaptivity of the system. The performance of our algorithm has been evaluated through the analysis of a P300 dataset provided by BCI Competition 2005 and another dataset collected from an in-house P300 speller system. The results show that our algorithm for joint feature selection and classification achieves satisfactory performance, meanwhile it can significantly reduce the training effort of the system. Furthermore, this algorithm is implemented online and the corresponding results demonstrate that our algorithm can improve the adaptiveness of the P300-based BCI speller.     

Effects of Correct and Wrong Answers on ERPs Recorded under Conditions of the Continuous Performance Test in ADHD/Normal Participants

Parameters of event-related potentials (ERPs) regarding correct and wrong answers under conditions of the continuous performance test (CPT) were measured in 50 adult subjects characterized by different levels of sustained attention with the absence/presence of attention deficit/hyperactivity disorders (ADHD). For ERP extraction, the average for each group of signals, which were time-locked to the onset of stimuli, was calculated; two ERP groups were considered separately for correct and wrong answers. In both groups, the P300 wave was clearly observed. The time dynamics of ERP components were investigated in six defined time blocks. At the peak of P300, a prominent component of brain activity could be observed. Some ERP morphological features (704 items) were extracted from these potentials. The results indicated that 11 of the obtained features showed a significant (P < 0.01) relation to the level of sustained attention. When comparing correct and wrong answers, 10 features in the normal group and 3 features in the ADHD group demonstrated significant differences (P < 0.05), which means that the participant’s response is reflected in the features of EEG signal. The results reveal a promising relation between CPT results and some parameters of brain signals, which can be used for further evaluations of the sustained attention level.     

Brain-Computer Interface Speller System for Alternative Communication: A Review

Brain-computer interface (BCI) speller is a system that provides an alternative communication for the disable people. The brain wave is translated into machine command through a BCI speller which can be used as a communication medium for the patients to express their thought without any motor movement. A BCI speller aims to spell characters by using the electroencephalogram (EEG) signal. Several types of BCI spellers are available based on the EEG signal. A standard BCI speller system consists of the following elements: BCI speller paradigm, data acquisition system and signal processing algorithms. In this work, a systematic review is provided on the BCI speller system and it includes speller paradigms, feature extraction, feature optimization and classification techniques for BCI speller. The advantages and limitations of different speller paradigm and machine learning algorithms are discussed in this article. Also, the future research directions are discussed which can overcome the limitations of present state-of-the-art techniques for BCI speller.     

Event related desynchronization: use as a neurophysiologic marker is restricted

The present research aims to show that the occurrence of alpha blocking or event-related desynchronization (ERD) strongly depends on the amplitude and also on the phase angle of alpha activity at the stimulus onset. Simple visual stimulation was presented to 17 healthy subjects during EEG recording. An O₂ electrode was used for analysis with a 32 channel EEG sampling system. We used a segmentation of raw data in order to obtain the evoked potential. Prestimulus and poststimulus activities were filtered in the alpha (8–13 Hz) frequency band. Later, four different events (blocked, time-locked, phase-locked, and eliminated) were separately averaged. Phase-locked sweeps were determined by application of inter-trial coherence analysis. The evaluation of the data shows that “time-locked and phase-locked sweeps” were the dominating pattern and not “the blocked pattern”, which occurred only when the prestimulus alpha was high. In the analyses of EEG-EP sweeps, only 22 % of epochs showed (ERD). The ANOVA revealed significant differences between four different alpha responses (F(3,48) = 11.175; p < 0.001). Furthermore, alpha oscillations in time-locked responses were significantly higher than blocked (p < 0.0001). The analyses clearly demonstrate that important precaution is needed when using the ERD as a cognitive or pathological marker.     

Functional Brain Connectivity as a New Feature for P300 Speller

The brain is a large-scale complex network often referred to as the “connectome”. Cognitive functions and information processing are mainly based on the interactions between distant brain regions. However, most of the ‘feature extraction’ methods used in the context of Brain Computer Interface (BCI) ignored the possible functional relationships between different signals recorded from distinct brain areas. In this paper, the functional connectivity quantified by the phase locking value (PLV) was introduced to characterize the evoked responses (ERPs) obtained in the case of target and non-targets visual stimuli. We also tested the possibility of using the functional connectivity in the context of ‘P300 speller’. The proposed approach was compared to the well-known methods proposed in the state of the art of “P300 Speller”, mainly the peak picking, the area, time/frequency based features, the xDAWN spatial filtering and the stepwise linear discriminant analysis (SWLDA). The electroencephalographic (EEG) signals recorded from ten subjects were analyzed offline. The results indicated that phase synchrony offers relevant information for the classification in a P300 speller. High synchronization between the brain regions was clearly observed during target trials, although no significant synchronization was detected for a non-target trial. The results showed also that phase synchrony provides higher performance than some existing methods for letter classification in a P300 speller principally when large number of trials is available. Finally, we tested the possible combination of both approaches (classical features and phase synchrony). Our findings showed an overall improvement of the performance of the P300-speller when using Peak picking, the area and frequency based features. Similar performances were obtained compared to xDAWN and SWLDA when using large number of trials.     

A Deep Learning Architecture for P300 Detection with Brain-Computer Interface Application

In this paper, a brain-computer interface (BCI) system for character recognition is proposed based on the P300 signal. A P300 speller is used to spell the word or character without any muscle movement. P300 detection is the first step to detect the character from the electroencephalogram (EEG) signal. The character is recognized from the detected P300 signal. In this paper, sparse autoencoder (SAE) and stacked sparse autoencoder (SSAE) based feature extraction methods are proposed for P300 detection. This work also proposes a fusion of deep-features with the temporal features for P300 detection. A SSAE technique extracts high-level information about input data. The combination of SSAE features with the temporal features provides abstract and temporal information about the signal. An ensemble of weighted artificial neural network (EWANN) is proposed for P300 detection to minimize the variation among different classifiers. To provide more importance to the good classifier for final classification, a higher weightage is assigned to the better performing classifier. These weights are calculated from the cross-validation test. The model is tested on two different publicly available datasets, and the proposed method provides better or comparable character recognition performance than the state-of-the-art methods.     

A comparison study of two P300 speller paradigms for brain–computer interface

In this paper, a comparison of two existing P300 spellers is conducted. In the first speller, the visual stimuli of characters are presented in a single character (SC) paradigm and each button corresponding to a character flashes individually in a random order. The second speller is based on a region-based (RB) paradigm. In the first level, all characters are grouped and each button corresponding to a group flashes individually in a random order. Once a group is selected, the characters in it will appear on the flashing buttons of the second level for the selection of desired character. In a spelling experiment involving 12 subjects, higher online accuracy was obtained on the RB paradigm-based P300 speller than the SC paradigm-based P300 speller. Furthermore, we analyzed P300 detection performance, the P300 waveforms and Fisher ratios using the data collected by the two spellers. It was found that the stimuli display paradigm of the RB speller enhances P300 potential and is more suitable for P300 detection.     

Channel Selection Based on Phase Measurement in P300-Based Brain-Computer Interface

Most EEG-based brain-computer interface (BCI) paradigms include specific electrode positions. As the structures and activities of the brain vary with each individual, contributing channels should be chosen based on original records of BCIs. Phase measurement is an important approach in EEG analyses, but seldom used for channel selections. In this paper, the phase locking and concentrating value-based recursive feature elimination approach (PLCV-RFE) is proposed to produce robust-EEG channel selections in a P300 speller. The PLCV-RFE, deriving from the phase resetting mechanism, measures the phase relation between EEGs and ranks channels by the recursive strategy. Data recorded from 32 electrodes on 9 subjects are used to evaluate the proposed method. The results show that the PLCV-RFE substantially reduces channel sets and improves recognition accuracies significantly. Moreover, compared with other state-of-the-art feature selection methods (SSNRSF and SVM-RFE), the PLCV-RFE achieves better performance. Thus the phase measurement is available in the channel selection of BCI and it may be an evidence to indirectly support that phase resetting is at least one reason for ERP generations.     

The P300 based brain-computer interface: effect of stimulus position in a stimulus train

The P300 brain-computer interface (BCI) is currently the most efficient BCI. This interface is based on detection of the P300 wave of the brain potentials evoked when a symbol related to the intended input is highlighted. To increase operation speed of the P300 BCI, reduction of the number of stimuli repetitions is needed. This reduction leads to increase of the relative contribution to the input symbol detection from the reaction to the first target stimulus. It is known that the event-related potentials (ERP) to the first stimulus presentations can be different from the ERP to stimuli presented latter. In particular, the amplitude of responses to the first stimulus presentations is often increased, which is beneficial for their recognition by the BCI. However, this effect was not studied within the BCI framework. The current study examined the ERP obtained from healthy participants (n = 14) in the standard P300 BCI paradigm using 10 trials, as well as in the modified P300 BCI with stimuli presented on moving objects in triple-trial (n = 6) and single-trial (n = 6) stimulation modes. Increased ERP amplitude was observed in response to the first target stimuli in both conditions, as well as in the single-trial mode comparing to triple-trial. We discuss the prospects of using the specific features of the ERP to first stimuli and the single-trial ERP for optimizing the high-speed modes in the P300 BCIs.     

Quantitative trait loci analysis of human event-related brain potentials: P3 voltage

The P3 event-related brain potential (ERP) is a positive-going voltage change of scalp-recorded electroencephalographic activity that occurs between 300–500 ms after stimulus onset. It is elicited when a stimulus is perceived, memory operations are engaged, and attentional resources are allocated toward its processing. Because this ERP component reflects fundamental cognitive processing, it has found wide utility as an assessment of human mental function in basic and clinical studies. In particular, P3 attributes are heritable and have demonstrated considerable promise as a means to identify individuals at genetic risk for alcoholism. We have conducted a quantitative linkage analysis on a large sample from families with a high density of affected individuals. The analyses suggest that several regions of the human genome contain genetic loci related to the generation of the P3 component of the ERP, which are possible candidate loci underlying the functional organization of human neuroelectric activity.     

Analysis of phase-locking is informative for studying event-related EEG activity

 A new method is presented for quantitative evaluation of single-sweep phase and amplitude electroencephalogram (EEG) characteristics that is a more informative approach in comparison with conventional signal averaging. In the averaged potential, phase-locking and amplitude effects of the EEG response cannot be separated. To overcome this problem, single-trial EEG sweeps are decomposed into separate presentations of their phase relationships and amplitude characteristics. The stability of the phase-coupling to stimulus is then evaluated independently by analyzing the single-sweep phase presentations. The method has the following advantages: information about stability of the phase-locking can be used to assess event-related oscillatory activity; the method permits evaluation of the timing of event-related phase-locking; and a global assessment and comparison of the phase-locking of ensembles of single sweeps elicited in different processing conditions is possible. The method was employed to study auditory alpha and theta responses in young and middle-aged adults. The results showed that whereas amplitudes of frequency responses tended to decrease, the phase-locking increased significantly with age. The synchronization with stimulus (phase-locking) was the only parameter reliably to differentiate the brain responses of the two age groups, as well as to reveal specific age-related changes in frontal evoked alpha activity. Thus, the present approach can be used to evaluate dynamic brain processes more precisely. Received: 12 February 1996 / Accepted in revised form: 11 October 1996     

Neuronal Correlates of a Virtual-Reality-Based Passive Sensory P300 Network

P300, a positive event-related potential (ERP) evoked at around 300 ms after stimulus, can be elicited using an active or passive oddball paradigm. Active P300 requires a person’s intentional response, whereas passive P300 does not require an intentional response. Passive P300 has been used in incommunicative patients for consciousness detection and brain computer interface. Active and passive P300 differ in amplitude, but not in latency or scalp distribution. However, no study has addressed the mechanism underlying the production of passive P300. In particular, it remains unclear whether the passive P300 shares an identical active P300 generating network architecture when no response is required. This study aims to explore the hierarchical network of passive sensory P300 production using dynamic causal modelling (DCM) for ERP and a novel virtual reality (VR)-based passive oddball paradigm. Moreover, we investigated the causal relationship of this passive P300 network and the changes in connection strength to address the possible functional roles. A classical ERP analysis was performed to verify that the proposed VR-based game can reliably elicit passive P300. The DCM results suggested that the passive and active P300 share the same parietal-frontal neural network for attentional control and, underlying the passive network, the feed-forward modulation is stronger than the feed-back one. The functional role of this forward modulation may indicate the delivery of sensory information, automatic detection of differences, and stimulus-driven attentional processes involved in performing this passive task. To our best knowledge, this is the first study to address the passive P300 network. The results of this study may provide a reference for future clinical studies on addressing the network alternations under pathological states of incommunicative patients. However, caution is required when comparing patients’ analytic results with this study. For example, the task presented here is not applicable to incommunicative patients.     

Prefrontal Neurons Encode Actions and Outcomes in Conjunction with Spatial Location in Rats Performing a Dynamic Delayed Non-Match to Position Task

To respond adaptively to change organisms must utilize information about recent events and environmental context to select actions that are likely to produce favorable outcomes. We developed a dynamic delayed nonmatching to position task to study the influence of spatial context on event-related activity of medial prefrontal cortex neurons during reinforcement-guided decision-making. We found neurons with responses related to preparation, movement, lever press responses, reinforcement, and memory delays. Combined event-related and video tracking analyses revealed variability in spatial tuning of neurons with similar event-related activity. While all correlated neurons exhibited spatial tuning broadly consistent with relevant task events, for instance reinforcement-related activity concentrated in locations where reinforcement was delivered, some had elevated activity in more specific locations, for instance reinforcement-related activity in one of several locations where reinforcement was delivered. Timing analyses revealed a limited set of distinct response types with activity time-locked to critical behavioral events that represent the temporal organization of dDNMTP trials. Our results suggest that reinforcement-guided decision-making emerges from discrete populations of medial prefrontal neurons that encode information related to planned or ongoing movements and actions and anticipated or actual action-outcomes in conjunction with information about spatial context.     

P300 from a single-stimulus paradigm: passive versus active tasks and stimulus modality

The P300 component of the event-related brain potential (ERP) was elicited with auditory and visual stimuli in separate experiments. Each study compared an oddball paradigm that presented both target and standard stimuli with a single-stimulus paradigm that presented a target but no standard stimuli. Subjects were instructed in different conditions either to ignore the stimuli, press a response key to the target, or maintain a mental count of the targets. For the passive ignore conditions, P300 amplitude from the single-stimulus paradigm was larger than that from the oddball paradigm. For the active tasks, P300 amplitude from the oddball paradigm was larger than that from the single-stimulus paradigm. For the press and count conditions, P300 amplitude and latency were highly similar for the oddball and single-stimulus procedures. The findings suggest that the single-stimulus paradigm can provide reliable cognitive measures in clinical/applied testing for both passive and active response conditions.     

A frequency-weighted method combined with Common Spatial Patterns for electroencephalogram classification in brain–computer interface

Common Spatial Patterns (CSP) has been proven to be a powerful and successful method in the detection of event-related desynchronization (ERD) and ERD based brain–computer interface (BCI). However, frequency optimization combined with CSP has only been investigated by a few groups. In this paper, a frequency-weighted method (FWM) is proposed to optimize the frequency spectrum of surface electroencephalogram (EEG) signals for a two-class mental task classification. This straightforward method computes a weight value for each frequency component according to its importance for the discrimination task and reforms the spectrum with the computed weights. The off-line analysis shows that the proposed method achieves an improvement of about 4% (averaged over 24 datasets) in terms of cross-validation accuracy over the basic CSP.     

Simultaneous Detection of P300 and Steady-State Visually Evoked Potentials for Hybrid Brain-Computer Interface

Objective

We study the feasibility of a hybrid Brain-Computer Interface (BCI) combining simultaneous visual oddball and Steady-State Visually Evoked Potential (SSVEP) paradigms, where both types of stimuli are superimposed on a computer screen. Potentially, such a combination could result in a system being able to operate faster than a purely P300-based BCI and encode more targets than a purely SSVEP-based BCI.

Approach

We analyse the interactions between the brain responses of the two paradigms, and assess the possibility to detect simultaneously the brain activity evoked by both paradigms, in a series of 3 experiments where EEG data are analysed offline.

Main Results

Despite differences in the shape of the P300 response between pure oddball and hybrid condition, we observe that the classification accuracy of this P300 response is not affected by the SSVEP stimulation. We do not observe either any effect of the oddball stimulation on the power of the SSVEP response in the frequency of stimulation. Finally results from the last experiment show the possibility of detecting both types of brain responses simultaneously and suggest not only the feasibility of such hybrid BCI but also a gain over pure oddball- and pure SSVEP-based BCIs in terms of communication rate.     

Classifying four-category visual objects using multiple ERP components in single-trial ERP

Object categorization using single-trial electroencephalography (EEG) data measured while participants view images has been studied intensively. In previous studies, multiple event-related potential (ERP) components (e.g., P1, N1, P2, and P3) were used to improve the performance of object categorization of visual stimuli. In this study, we introduce a novel method that uses multiple-kernel support vector machine to fuse multiple ERP component features. We investigate whether fusing the potential complementary information of different ERP components (e.g., P1, N1, P2a, and P2b) can improve the performance of four-category visual object classification in single-trial EEGs. We also compare the classification accuracy of different ERP component fusion methods. Our experimental results indicate that the classification accuracy increases through multiple ERP fusion. Additional comparative analyses indicate that the multiple-kernel fusion method can achieve a mean classification accuracy higher than 72 %, which is substantially better than that achieved with any single ERP component feature (55.07 % for the best single ERP component, N1). We compare the classification results with those of other fusion methods and determine that the accuracy of the multiple-kernel fusion method is 5.47, 4.06, and 16.90 % higher than those of feature concatenation, feature extraction, and decision fusion, respectively. Our study shows that our multiple-kernel fusion method outperforms other fusion methods and thus provides a means to improve the classification performance of single-trial ERPs in brain–computer interface research.     

A computational model of cellular mechanisms of temporal coding in the medial geniculate body (MGB)

Acoustic stimuli are often represented in the early auditory pathway as patterns of neural activity synchronized to time-varying features. This phase-locking predominates until the level of the medial geniculate body (MGB), where previous studies have identified two main, largely segregated response types: Stimulus-synchronized responses faithfully preserve the temporal coding from its afferent inputs, and Non-synchronized responses, which are not phase locked to the inputs, represent changes in temporal modulation by a rate code. The cellular mechanisms underlying this transformation from phase-locked to rate code are not well understood. We use a computational model of a MGB thalamocortical neuron to test the hypothesis that these response classes arise from inferior colliculus (IC) excitatory afferents with divergent properties similar to those observed in brain slice studies. Large-conductance inputs exhibiting synaptic depression preserved input synchrony as short as 12.5 ms interclick intervals, while maintaining low firing rates and low-pass filtering responses. By contrast, small-conductance inputs with Mixed plasticity (depression of AMPA-receptor component and facilitation of NMDA-receptor component) desynchronized afferent inputs, generated a click-rate dependent increase in firing rate, and high-pass filtered the inputs. Synaptic inputs with facilitation often permitted band-pass synchrony along with band-pass rate tuning. These responses could be tuned by changes in membrane potential, strength of the NMDA component, and characteristics of synaptic plasticity. These results demonstrate how the same synchronized input spike trains from the inferior colliculus can be transformed into different representations of temporal modulation by divergent synaptic properties.     

Neural synchrony during response production and inhibition

Inhibition of irrelevant information (conflict monitoring) and/or of prepotent actions is an essential component of adaptive self-organized behavior. Neural dynamics underlying these functions has been studied in humans using event-related brain potentials (ERPs) elicited in Go/NoGo tasks that require a speeded motor response to the Go stimuli and withholding a prepotent response when a NoGo stimulus is presented. However, averaged ERP waveforms provide only limited information about the neuronal mechanisms underlying stimulus processing, motor preparation, and response production or inhibition. In this study, we examine the cortical representation of conflict monitoring and response inhibition using time-frequency analysis of electroencephalographic (EEG) recordings during continuous performance Go/NoGo task in 50 young adult females. We hypothesized that response inhibition would be associated with a transient boost in both temporal and spatial synchronization of prefrontal cortical activity, consistent with the role of the anterior cingulate and lateral prefrontal cortices in cognitive control. Overall, phase synchronization across trials measured by Phase Locking Index and phase synchronization between electrode sites measured by Phase Coherence were the highest in the Go and NoGo conditions, intermediate in the Warning condition, and the lowest under Neutral condition. The NoGo condition was characterized by significantly higher fronto-central synchronization in the 300-600 ms window, whereas in the Go condition, delta- and theta-band synchronization was higher in centro-parietal regions in the first 300 ms after the stimulus onset. The present findings suggest that response production and inhibition is supported by dynamic functional networks characterized by distinct patterns of temporal and spatial synchronization of brain oscillations.     

DIURNAL VARIATION IN THE P300 COMPONENT OF HUMAN COGNITIVE EVENT-RELATED POTENTIAL

Diurnal variation in the P300 component of the human cognitive event-related potential (ERP) was examined. The P300 component is considered to be a measure of neuroelectric activity related to cognitive functions such as attention allocation and information processing. Nine diurnally active healthy male subjects whose sleep-wake rhythms were synchronized prior to the experiment were studied. The P300 components oral temperature, heart rate, left- and right-hand grip strength, reaction time, subjectively rated sleepiness, attention level, and fatigue were measured at 08:00, 11:00, 14:00, 17:00, and 20:00. Significant diurnal variations in P300 latency, P300 amplitude, oral temperature, heart rate, left- and right-hand grip strength, subjectively rated sleepiness, and attention level were observed. The P300 latency at 08:00 was significantly longer than at 11:00, 17:00, and 20:00, while the P300 amplitude at 08:00 was significantly greater than at 17:00 and 20:00. The P300 latency was correlated positively with subjectively rated sleepiness and negatively correlated with subjectively rated attention level. These results suggest the existence of diurnal variation in human cognitive functions. (Chronobiology International, 17(5), 669–678, 2000)