<Emphasis Type="Italic">P</Emphasis>300-based brain-computer interface: The effect of the stimulus position in a stimulus train

The most popular type of brain-computer interfaces (BCIs) are based on the detection of the P300 wave of the evoked potentials appearing in response to a stimulus chosen by the subject. In order to increase the speed of operation of these BCIs, it is possible to decrease the number of repeated stimulus presentations. It is associated with an increase in the relative importance of the response to the first stimulus in a train for correct recognition of the stimulus chosen. Event-related potentials (ERPs) in response to the first stimulus presentations are known to have their own specificity. Particularly, in many cases, the amplitude of the response to the first presentations is enhanced, which makes it very suitable for recognition in a BCI. However, this effect has not been studied to date. In this study, the ERPs recorded in healthy subjects in a standard BCI paradigm (n = 14) with ten presentations of stimuli or during triple-trial (n = 6) and single-trial (n = 6) presentations of stimuli in a modified BCI paradigm with moving objects have been analyzed. In both cases, first presentations of the target stimuli or single-trial presentation of the target stimulus were associated with higher amplitudes of ERPs. The opportunity to use specific differences between the responses to the first or single-trial presentations and the responses to later stimuli during their repeated presentations for improving high-speed operations in the P300-based BCI is discussed.     

A P300-based Brain-Computer Interface with Stimuli on Moving Objects: Four-Session Single-Trial and Triple-Trial Tests with a Game-Like Task Design

Brain-computer interfaces (BCIs) are tools for controlling computers and other devices without using muscular activity, employing user-controlled variations in signals recorded from the user’s brain. One of the most efficient noninvasive BCIs is based on the P300 wave of the brain’s response to stimuli and is therefore referred to as the P300 BCI. Many modifications of this BCI have been proposed to further improve the BCI’s characteristics or to better adapt the BCI to various applications. However, in the original P300 BCI and in all of its modifications, the spatial positions of stimuli were fixed relative to each other, which can impose constraints on designing applications controlled by this BCI. We designed and tested a P300 BCI with stimuli presented on objects that were freely moving on a screen at a speed of 5.4°/s. Healthy participants practiced a game-like task with this BCI in either single-trial or triple-trial mode within four sessions. At each step, the participants were required to select one of nine moving objects. The mean online accuracy of BCI-based selection was 81% in the triple-trial mode and 65% in the single-trial mode. A relatively high P300 amplitude was observed in response to targets in most participants. Self-rated interest in the task was high and stable over the four sessions (the medians in the 1st/4th sessions were 79/84% and 76/71% in the groups practicing in the single-trial and triple-trial modes, respectively). We conclude that the movement of stimulus positions relative to each other may not prevent the efficient use of the P300 BCI by people controlling their gaze, e.g., in robotic devices and in video games.     

An Efficient ERP-Based Brain-Computer Interface Using Random Set Presentation and Face Familiarity

Event-related potential (ERP)-based P300 spellers are commonly used in the field of brain-computer interfaces as an alternative channel of communication for people with severe neuro-muscular diseases. This study introduces a novel P300 based brain-computer interface (BCI) stimulus paradigm using a random set presentation pattern and exploiting the effects of face familiarity. The effect of face familiarity is widely studied in the cognitive neurosciences and has recently been addressed for the purpose of BCI. In this study we compare P300-based BCI performances of a conventional row-column (RC)-based paradigm with our approach that combines a random set presentation paradigm with (non-) self-face stimuli. Our experimental results indicate stronger deflections of the ERPs in response to face stimuli, which are further enhanced when using the self-face images, and thereby improving P300-based spelling performance. This lead to a significant reduction of stimulus sequences required for correct character classification. These findings demonstrate a promising new approach for improving the speed and thus fluency of BCI-enhanced communication with the widely used P300-based BCI setup.     

Estimating the Intended Sound Direction of the User: Toward an Auditory Brain-Computer Interface Using Out-of-Head Sound Localization

The auditory Brain-Computer Interface (BCI) using electroencephalograms (EEG) is a subject of intensive study. As a cue, auditory BCIs can deal with many of the characteristics of stimuli such as tone, pitch, and voices. Spatial information on auditory stimuli also provides useful information for a BCI. However, in a portable system, virtual auditory stimuli have to be presented spatially through earphones or headphones, instead of loudspeakers. We investigated the possibility of an auditory BCI using the out-of-head sound localization technique, which enables us to present virtual auditory stimuli to users from any direction, through earphones. The feasibility of a BCI using this technique was evaluated in an EEG oddball experiment and offline analysis. A virtual auditory stimulus was presented to the subject from one of six directions. Using a support vector machine, we were able to classify whether the subject attended the direction of a presented stimulus from EEG signals. The mean accuracy across subjects was 70.0% in the single-trial classification. When we used trial-averaged EEG signals as inputs to the classifier, the mean accuracy across seven subjects reached 89.5% (for 10-trial averaging). Further analysis showed that the P300 event-related potential responses from 200 to 500 ms in central and posterior regions of the brain contributed to the classification. In comparison with the results obtained from a loudspeaker experiment, we confirmed that stimulus presentation by out-of-head sound localization achieved similar event-related potential responses and classification performances. These results suggest that out-of-head sound localization enables us to provide a high-performance and loudspeaker-less portable BCI system.     

Visual Stimuli for P300-Based Brain-Computer Interfaces: Color,Shape, and Mobility

The purpose of this study was to identify the impact of different discriminative features of stimuli in a P300 brain-computer interface paradigm on overall performance and evoked potentials. It has been shown that stimuli sets with a greater number of discriminative features yield better target selection accuracy. Target selection accuracy was significantly higher for the stimuli that differ from each other by color, shape, and semantics. Highest performance was achieved with the stimuli set containing the largest number of discriminative features, namely a set of nine different-colored letters. This result is mainly due to higher mean P300 peak amplitude for stimuli sets that contain more discriminative features. The results of the study can be used for designing a better user experience in brain-computer interfacing (BCI). Motion of the stimuli presentation point and characteristics of this motion (linear or pseudorandom) did not have any impact on BCI performance. This result is promising for future BCI designs with rapid serial visual presentation using mobile robots or augmented reality as stimuli presentation environment.     

P300, stimulus intensity,and modality

Auditory and visual stimulus intensity levels were manipulated systematically in separate conditions to assess the influence of these variables on the P300 event-related brain potential (ERP). Increases in stimulus intensity produced increases in P300 amplitude and decreases in peak latency for both modalities, although the latency effects were stronger for visual stimulation. Similar, somewhat weaker stimulus intensity effects also were observed for the N100, P200, and N200 components. The findings suggest that stimulus intensity contributes to both P300 amplitude and latency measures in important ways and are discussed in relation to the use of ERPs in applied contexts.     

Characteristics of event-related potentials in response to symbolical and alphabetical stimulation matrices used in a P300-based brain-computer interface

In order to create a P300-based brain-computer interface (BCI) (the so-called Farwell-Donchin paradigm, FD) with a symbol matrix used as a stimulus, we compared characteristics of event-related potentials (ERPs) in response to stimulation by 6 × 6 matrices composed of either pictogram symbols or Cyrillic alphabet characters. Nine healthy adults were examined in 18 experiments, during which 28-channel EEGs were recorded in the course of stimulation with matrices of these two types. The obtained ERP data, i.e., amplitudes and peak latencies of the ERP components N1, P3 (with the P3a and P3b sub-components), and N4 were compared and analyzed for different types of stimulation matrices. In at least seven out of nine subjects, P3a, P3b, and N4 ERP amplitudes were larger in response to the symbol matrix than to the character matrix, while N1 amplitudes were larger for the character matrix. For N1 and P3a, the ERP latencies were shorter for the symbol matrix, while for P3b and N4, they were longer for the character matrix. The topography of differential ERP responses to the two types of stimuli was analyzed using a series of paired t-tests. Differences of ERP component amplitudes were determined individually for each of the 28 channels; next, for each site, absolute t-test values were summed for all nine subjects. For all ERP components studied, the t-test for peak amplitudes in response to target and non-target letters identified two separate areas with distinct lateralization. ERP responses to target and non-target symbols differed most in transversely extended areas. Finally, the yield surface of differential response to target letters and target symbols had a complex topography.     

P300 from a single auditory stimulus

The P3(00) event-related brain potential (ERP) was elicited with auditory stimuli to compare 2 different discrimination tasks. The oddball paradigm presented both target and standard tones; the single-stimulus paradigm presented at target but no standard tone stimulus. Experiment 1 manipulated target stimulus probability (0.20, 0.50, 0.80) and produced highly similar P3 amplitude and latency results across probability levels for each paradigm. Experiment 2 factorially varied inter-stimulus interval (2 sec, 6 sec) and target stimulus probability (0.20, 0.80). P3 amplitude and latency were highly similar for both the oddball and single-stimulus procedures across all conditions.     

视觉空间注意事件相关电位的协同学分析

在用事件相关电位(event-related potentials,ERP)研究视觉空间注意问题时,直接观察ERP数据就可得出,空间注意的主要作用是对视觉信息处理的调制,它出现在刺激开始后大约80～250ms,主要表现为枕叶的P1、N1和P2波有明显的增强但它们的潜伏期没有变化。采用基于协同学的时空模式分解方法,把视觉空间注意ERP分解为3个模式成分。结果表明,注意不仅使模式1的第一个正波成分(P11)、第一个负波成分(N11)以及第二个正波成分(P12)增强,还使模式3的第一个正波成分(P31)的潜伏期缩短。用探照灯模型对这些现象作了初步解释,说明该方法是研究注意ERP的一种有潜力的新方法。     

P300 from a single-stimulus paradigm: passive versus active tasks and stimulus modality

The P300 component of the event-related brain potential (ERP) was elicited with auditory and visual stimuli in separate experiments. Each study compared an oddball paradigm that presented both target and standard stimuli with a single-stimulus paradigm that presented a target but no standard stimuli. Subjects were instructed in different conditions either to ignore the stimuli, press a response key to the target, or maintain a mental count of the targets. For the passive ignore conditions, P300 amplitude from the single-stimulus paradigm was larger than that from the oddball paradigm. For the active tasks, P300 amplitude from the oddball paradigm was larger than that from the single-stimulus paradigm. For the press and count conditions, P300 amplitude and latency were highly similar for the oddball and single-stimulus procedures. The findings suggest that the single-stimulus paradigm can provide reliable cognitive measures in clinical/applied testing for both passive and active response conditions.     

Exploring Combinations of Auditory and Visual Stimuli for Gaze-Independent Brain-Computer Interfaces

For Brain-Computer Interface (BCI) systems that are designed for users with severe impairments of the oculomotor system, an appropriate mode of presenting stimuli to the user is crucial. To investigate whether multi-sensory integration can be exploited in the gaze-independent event-related potentials (ERP) speller and to enhance BCI performance, we designed a visual-auditory speller. We investigate the possibility to enhance stimulus presentation by combining visual and auditory stimuli within gaze-independent spellers. In this study with N = 15 healthy users, two different ways of combining the two sensory modalities are proposed: simultaneous redundant streams (Combined-Speller) and interleaved independent streams (Parallel-Speller). Unimodal stimuli were applied as control conditions. The workload, ERP components, classification accuracy and resulting spelling speed were analyzed for each condition. The Combined-speller showed a lower workload than uni-modal paradigms, without the sacrifice of spelling performance. Besides, shorter latencies, lower amplitudes, as well as a shift of the temporal and spatial distribution of discriminative information were observed for Combined-speller. These results are important and are inspirations for future studies to search the reason for these differences. For the more innovative and demanding Parallel-Speller, where the auditory and visual domains are independent from each other, a proof of concept was obtained: fifteen users could spell online with a mean accuracy of 87.7% (chance level <3%) showing a competitive average speed of 1.65 symbols per minute. The fact that it requires only one selection period per symbol makes it a good candidate for a fast communication channel. It brings a new insight into the true multisensory stimuli paradigms. Novel approaches for combining two sensory modalities were designed here, which are valuable for the development of ERP-based BCI paradigms.     

Analysis of event-related potentials to verbal stimuli in healthy subjects and schizophrenia patients

In 30 healthy subjects and 32 patients after the first episode of schizophrenia 19 channel-EEG was recorded during visual presentation of a random sequence of words and pseudo-words. In the first series of the experiments, subjects had to read the presented verbal stimuli, in the second series they had to press a button when seeing a word, and in the third series they were instructed to press the button when seeing a pseudo-word. We studied components N170, P300 and N400. In the group of healthy subjects, the amplitude of N170 increased to words in the situation of their relevance, which corresponds to the "recognition potential", whereas in the group of patients, the amplitude of N170 increased to pseudo-words when they were relevant. So it was a paradoxical response. The amplitude of the ERP later waves (P300 and N400) in the group of schizophrenic patients was smaller and the relevance effect was impaired when the target stimuli were pseudo-words. However, the incongruity effect consisting in an increase in N400 amplitude to a non-target stimulus remained intact in patients.     

Time-locked and phase-locked features of P300 event-related potentials (ERPs) for brain–computer interface speller

The brain–computer interface P300 speller is aimed to help those patients unable to activate muscles to spell words by utilizing their brain activity. However, a problem associated with the use of this brain–computer interface paradigm is the generation mechanics of P300 related to responses to visual stimuli. Herein, we investigated the event-related potential (ERP) response for the P300-based brain–computer interface speller. A signal preprocessing method integrated coherent average, principal component analysis (PCA) and independent component analysis (ICA) to reduce the dimensions and noise in the raw data. The time–frequency analysis was based on wavelet and two characteristic parameters of event-related spectral perturbation (ERSP) and inter-trial coherence (ITC) were computed to indicate the evoked response (time-locked) and phase reset (phase-locked) activity, respectively. Results demonstrated that the proposed method was valid for the time-locked and phase-locked feature extraction and both the evoked response and phase reset contributed to the genesis of the P300 signal. These electrophysiological responses characteristics of ERPs would be used for BCI P300 speller design and its signal processing strategies.     

The habituation of event-related potentials to speech sounds and tones

We examined the short- and long-term habituation of auditory event-related potentials (ERPs) elicited by tones, complex tones and digitized speech sounds (vowels and consonant-vowel-consonant syllables). Twelve different stimuli equated in loudness and duration (300 msec) were studied. To examine short-term habituation stimuli were presented in trains of 6 with interstimulus intervals of 0.5 or 1.0 sec. The first 4 stimuli in a train were identical standards. On 50% of the trains the standard in the 5th position was replaced by a deviant probe stimulus, and on 20% of the trains the standard in the 6th position was replaced by a target, a truncated standard that required a speeded button press response.Short-term habituation (STH) was complete by the third stimulus in the train and resulted in amplitude decrements of 50–75% for the N1 component. STH was partially stimulus specific in that amplitudes were larger following deviant stimuli in the 5th position than following standards. STH of the N1 was more marked for speech sounds than for loudness-matched tones or complex tones at short ISI. In addition, standard and deviant stimuli that differed in phonetic structure showed more cross-habituation than did tones or complex tones that differed in frequency. This pattern of results suggests that STH is a function of the acoustic resemblance of successive stimuli.The long-term habituation (LTH) of the ERP was studied by comparing amplitudes across balanced 5.25 m stimulus blocks over the course of the experiment. Two types of LTH were observed. The N1 showed stimulus-specific LTH in that N1 amplitudes declined during the presentation of a stimulus, but returned to control levels when a different stimulus was presented in the subsequent condition. In contrast, the P3 elicited by the deviant stimuli showed non-specific LTH, being reduced across successive blocks containing different stimuli. P3s elicited by target stimuli remained stable in amplitude.     

Single-trial ERP evidence for the three-stage scheme of facial expression processing

Using a rapid serial visual presentation paradigm, we previously showed that the average amplitudes of six event-related potential (ERP) components were affected by different categories of emotional faces. In the current study, we investigated the six discriminating components on a single-trial level to clarify whether the amplitude difference between experimental conditions results from a difference in the real variability of single-trial amplitudes or from latency jitter across trials. It is found that there were consistent amplitude differences in the single-trial P1, N170, VPP, N3, and P3 components, demonstrating that a substantial proportion of the average amplitude differences can be explained by the pure variability in amplitudes on a single-trial basis between experimental conditions. These single-trial results verified the three-stage scheme of facial expression processing beyond multitrial ERP averaging, and showed the three processing stages of "fear popup", "emotional/unemotional discrimination", and "complete separation" based on the single-trial ERP dynamics.     

Effects of attention and slow potential shifts on self-regulation of event-related potentials

Research on the effects of self-regulation of slow potentials (SP) and event-related potentials (ERP) has failed to look at the possible interactions of these two kinds of brain potentials. The present study investigated such interactions by recording both ERP and SP potential changes in an operant ERP conditioning paradigm. Ten subjects participated in two conditions that were designed to differentially manipulate attention to the stimuli. In the operant conditioning task, subjects received auditory feedback as they attempted to increase the ERP amplitude at 180 msec poststimulus (P180), which was elicited by a subpainful shock stimulus to the forearm over 250 trials. In the distraction task, subjects were instructed not to attend to stimuli or feedback tones, but rather received and were tested on reading materials. Attention, as manipulated by these tasks, was not a determinant of changes in ERP amplitude since there were no significant differences in the size of P180 between attention conditions. While no significant change in the mean ERP amplitude occurred, subjects were able to produce ERPs above criterion threshold significantly more often during trials in the conditioning task than in the reading task. Thus, there was evidence of some learning. The difference in wave forms between hit and miss trials indicates a latency shift (with misses having a later ERP peak). This may indicate that latency, rather than, or in addition to, amplitude, is shaped during conditioning procedures. In addition, the CNV that developed between the shock stimulus and the feedback signal during conditioning was significantly larger in amplitude than in the distraction condition. This is taken as evidence of increased attention during conditioning. Since hit trials demonstrated larger contingent negative variation (CNV) amplitudes, production of CNVs may be instrumental in mediating hits. Therefore, attentional mechanisms may play a role in successful ERP self-regulation. No correlations were found involving P180, CNVs, or tonic slow potential shifts. Changes in tonic DC levels showed a suggestive trend between conditions. Although both conditions began with a negative shift, during conditioning the negativity increased, while during distraction the tonic level went to positivity. These trends support the hypothesis that attention and arousal increased during conditioning. The possible reasons for the lack of significant correlations between ERP and tonic or phasic slow potential changes in this paradigm are discussed.     

Single-event-related potential analysis by means of fragmentary decomposition

 A recently developed fragmentary decomposition method is employed to analyse single-trial event-related potentials (ERPs), thereby extending the traditional method of averaging. Using a conventional auditory oddball paradigm with 40 target stimuli, single-trial ERPs in 40 normal subjects were analysed for midline scalp (Fz, Cz and Pz) recording sites. The normalization effect, reported in our previous study of eye blink EMGs and proposed to be a characteristic property of a wide class of non-stationary physiological processes, was found to apply to these single-trial ERPs. Fragmentary decomposition of single-trial ERPs may be regarded as re-statement of the normalization effect. This allows both pre-stimulus EEGs and post-stimulus ERPs to be regarded as overlapping generic mass potentials (GMPs), with a characteristic Gaussian amplitude spectrum. On theoretical and empirical grounds we uniquely deduce a model GMP using an introduced “bud” function, and physically support it by the resting and transient conditions. The model takes into account the shape of the component, which suggests a simple relationship between the peak latency and the time of the component onset. Given that GMPs may be manipulated and sorted out, we present principles of the fragmentary synthesis, i.e. probabilistic ERP reconstructions on the basis of individual and ensemble properties of its identified components. Summarizing the component quantification in the form of the dynamic model provides for the first time the opportunity to quantify all significant components in single-trial ERPs. This method of single-trial analysis opens up new possibilities of exploring the dynamical ERP changes within a recording trial, particularly in late component “cognitive” paradigms. Received: 29 August 2000 / Accepted in revised form: 5 February 2001     

Prediction of Auditory and Visual P300 Brain-Computer Interface Aptitude

Objective

Brain-computer interfaces (BCIs) provide a non-muscular communication channel for patients with late-stage motoneuron disease (e.g., amyotrophic lateral sclerosis (ALS)) or otherwise motor impaired people and are also used for motor rehabilitation in chronic stroke. Differences in the ability to use a BCI vary from person to person and from session to session. A reliable predictor of aptitude would allow for the selection of suitable BCI paradigms. For this reason, we investigated whether P300 BCI aptitude could be predicted from a short experiment with a standard auditory oddball.

Methods

Forty healthy participants performed an electroencephalography (EEG) based visual and auditory P300-BCI spelling task in a single session. In addition, prior to each session an auditory oddball was presented. Features extracted from the auditory oddball were analyzed with respect to predictive power for BCI aptitude.

Results

Correlation between auditory oddball response and P300 BCI accuracy revealed a strong relationship between accuracy and N2 amplitude and the amplitude of a late ERP component between 400 and 600 ms. Interestingly, the P3 amplitude of the auditory oddball response was not correlated with accuracy.

Conclusions

Event-related potentials recorded during a standard auditory oddball session moderately predict aptitude in an audiory and highly in a visual P300 BCI. The predictor will allow for faster paradigm selection.

Significance

Our method will reduce strain on patients because unsuccessful training may be avoided, provided the results can be generalized to the patient population.     

Single-trial detection for intraoperative somatosensory evoked potentials monitoring

Abnormalities of somatosensory evoked potentials (SEPs) provide effective evidence for impairment of the somatosensory system, so that SEPs have been widely used in both clinical diagnosis and intraoperative neurophysiological monitoring. However, due to their low signal-to-noise ratio (SNR), SEPs are generally measured using ensemble averaging across hundreds of trials, thus unavoidably producing a tardiness of SEPs to the potential damages caused by surgical maneuvers and a loss of dynamical information of cortical processing related to somatosensory inputs. Here, we aimed to enhance the SNR of single-trial SEPs using Kalman filtering and time–frequency multiple linear regression (TF-MLR) and measure their single-trial parameters, both in the time domain and in the time–frequency domain. We first showed that, Kalman filtering and TF-MLR can effectively capture the single-trial SEP responses and provide accurate estimates of single-trial SEP parameters in the time domain and time–frequency domain, respectively. Furthermore, we identified significant correlations between the stimulus intensity and a set of indicative single-trial SEP parameters, including the correlation coefficient (between each single-trial SEPs and their average), P37 amplitude, N45 amplitude, P37-N45 amplitude, and phase value (at the zero-crossing points between P37 and N45). Finally, based on each indicative single-trial SEP parameter, we investigated the minimum number of trials required on a single-trial basis to suggest the existence of SEP responses, thus providing important information for fast SEP extraction in intraoperative monitoring.     

Impaired mismatch negativity (MMN) generation in schizophrenia as a function of stimulus deviance,probability, and interstimulus/interdeviant interval

Schizophrenia is a severe mental disorder associated with disturbances in perception and cognition. Event-related potentials (ERP) provide a mechanism for evaluating potential mechanisms underlying neurophysiological dysfunction in schizophrenia. Mismatch negativity (MMN) is a short-duration auditory cognitive ERP component that indexes operation of the auditory sensory (`echoic') memory system. Prior studies have demonstrated impaired MMN generation in schizophrenia along with deficits in auditory sensory memory performance. MMN is elicited in an auditory oddball paradigm in which a sequence of repetitive standard tones is interrupted infrequently by a physically deviant (`oddball') stimulus. The present study evaluates MMN generation as a function of deviant stimulus probability, interstimulus interval, interdeviant interval and the degree of pitch separation between the standard and deviant stimuli. The major findings of the present study are first, that MMN amplitude is decreased in schizophrenia across a broad range of stimulus conditions, and second, that the degree of deficit in schizophrenia is largest under conditions when MMN is normally largest. The pattern of deficit observed in schizophrenia differs from the pattern observed in other conditions associated with MMN dysfunction, including Alzheimer's disease, stroke, and alcohol intoxication.