A Co-Adaptive Brain-Computer Interface for End Users with Severe Motor Impairment

Co-adaptive training paradigms for event-related desynchronization (ERD) based brain-computer interfaces (BCI) have proven effective for healthy users. As of yet, it is not clear whether co-adaptive training paradigms can also benefit users with severe motor impairment. The primary goal of our paper was to evaluate a novel cue-guided, co-adaptive BCI training paradigm with severely impaired volunteers. The co-adaptive BCI supports a non-control state, which is an important step toward intuitive, self-paced control. A secondary aim was to have the same participants operate a specifically designed self-paced BCI training paradigm based on the auto-calibrated classifier. The co-adaptive BCI analyzed the electroencephalogram from three bipolar derivations (C3, Cz, and C4) online, while the 22 end users alternately performed right hand movement imagery (MI), left hand MI and relax with eyes open (non-control state). After less than five minutes, the BCI auto-calibrated and proceeded to provide visual feedback for the MI task that could be classified better against the non-control state. The BCI continued to regularly recalibrate. In every calibration step, the system performed trial-based outlier rejection and trained a linear discriminant analysis classifier based on one auto-selected logarithmic band-power feature. In 24 minutes of training, the co-adaptive BCI worked significantly (p = 0.01) better than chance for 18 of 22 end users. The self-paced BCI training paradigm worked significantly (p = 0.01) better than chance in 11 of 20 end users. The presented co-adaptive BCI complements existing approaches in that it supports a non-control state, requires very little setup time, requires no BCI expert and works online based on only two electrodes. The preliminary results from the self-paced BCI paradigm compare favorably to previous studies and the collected data will allow to further improve self-paced BCI systems for disabled users.     

Evaluation of EEG Oscillatory Patterns and Cognitive Process during Simple and Compound Limb Motor Imagery

Motor imagery (MI), sharing similar neural representations to motor execution, is regarded as a window to investigate the cognitive motor processes. However, in comparison to simple limb motor imagery, significantly less work has been reported on brain oscillatory patterns induced by compound limb motor imagery which involves several parts of limbs. This study aims to investigate differences of the electroencephalogram (EEG) patterns as well as cognitive process between simple limb motor imagery and compound limb motor imagery. Ten subjects participated in the experiment involving three tasks of simple limb motor imagery (left hand, right hand, feet) and three tasks of compound limb motor imagery (both hands, left hand combined with right foot, right hand combined with left foot). Simultaneous imagination of different limbs contributes to the activation of larger cortical areas as well as two estimated sources located at corresponding motor areas within beta rhythm. Compared with simple limb motor imagery, compound limb motor imagery presents a network with more effective interactions overlying larger brain regions, additionally shows significantly larger causal flow over sensorimotor areas and larger causal density over both sensorimotor areas and neighboring regions. On the other hand, compound limb motor imagery also shows significantly larger 10–11 Hz alpha desynchronization at occipital areas and central theta synchronization. Furthermore, the phase-locking value (PLV) between central and occipital areas of left/right hand combined with contralateral foot imagery is significantly larger than that of simple limb motor imagery. All these findings imply that there exist apparent intrinsic distinctions of neural mechanism between simple and compound limb motor imagery, which presents a more complex effective connectivity network and may involve a more complex cognitive process during information processing.     

Localization of brain electrical activity sources and hemodynamic activity foci during motor imagery

The sources of brain activity that make the maximum contribution to EEG patterns corresponding to motor imagery have been studied. The accuracy of their classification determines the efficiency of brain-computer interface (BCI) for controlling external technical devices directly by brain signals, without the involvement of muscle activity. Brain activity sources are identified by independent component analysis. The independent components providing the maximum BCI classification accuracy are considered relevant for the motor imagery task. The two most relevant sources exhibit clearly marked event-related desynchronization and synchronization of the μ-rhythm during the imagery of contra- and ipsilateral hand movements. These sources were localized by solving the inverse EEG problem with due consideration for individual geometry of the brain and its covers, as determined by magnetic resonance imaging. Each of the sources was shown to be localized in the 3a area of the primary somatosensory cortex corresponding to proprioceptive sensitivity of the contralateral hand. Their positions were close to the foci of BOLD activity obtained by fMRI.     

The Efficiency of the Brain-Computer Interfaces Based on Motor Imagery with Tactile and Visual Feedback

In this study we compared tactile and visual feedbacks for the motor imagery-based brain–computer interface (BCI) in five healthy subjects. A vertical green bar from the center of the fixing cross to the edge of the screen was used as visual feedback. Vibration motors that were placed on the forearms of the right and the left hands and on the back of the subject’s neck were used as tactile feedback. A vibration signal was used to confirm the correct classification of the EEG patterns of the motor imagery of right and left hand movements and the rest task. The accuracy of recognition in the classification of the three states (right hand movement, left hand movement, and rest) in the BCI without feedback exceeded the random level (33% for the three states) for all the subjects and was rather high (67.8% ± 13.4% (mean ± standard deviation)). Including the visual and tactile feedback in the BCI did not significantly change the mean accuracy of recognition of mental states for all the subjects (70.5% ± 14.8% for the visual feedback and 65.9% ± 12.4% for the tactile feedback). The analysis of the dynamics of the movement imagery skill in BCI users with the tactile and visual feedback showed no significant differences between these types of feedback. Thus, it has been found that the tactile feedback can be used in the motor imagery-based BCI instead of the commonly used visual feedback, which greatly expands the possibilities of the practical application of the BCI.     

Principles of neurorehabilitation based on the brain-computer interface and biologically adequate control of the exoskeleton

The neurophysiological prerequisites for the development and operation of the brain-computer interfaces (BCI) that allow cerebral electrical signals alone to control external technical devices are considered. A BCI based on the discrimination of the EEG patterns related to imagery of extremity movements is described. The possibility of the rehabilitation of patients with motor disorders by means of the BCI based on motor imagery and the exoskeleton controlled by it is discussed.     

The comparison of motor learning performance with and without feedback

Ten individuals were divided into two feedback and no-feedback groups. The effect of abstract visual feedback was investigated in these two groups. Using eight electroencephalography (EEG) electrodes, the induced event-related desynchronization/synchronization of the EEG of three motor imagery tasks (left hand, right hand, and right foot) was analyzed by wavelet and spatial filtering methods. Linear discriminant analysis was used to classify the three imagery tasks. Each imagery task's total length was set to 3?s and 1?s of it was used for the classification. The classification result was shown to the subjects of the feedback group in a real-time manner as an abstract visual feedback. While the paired t-test of the first and third sessions of the training days confirmed the improvement of the motor imagery learning in the feedback group (p?<?0.01), the motor imagery learning of the no-feedback group was not significant.     

EEG and behavioral changes in a hyperkinetic child concurrent with training of the sensorimotor rhythm (SMR)

Reduced seizure incidence coupled with voluntary motor inhibition accompanied conditioned increases in the sensorimotor rhythm(SMR), a 12–14 Hz rhythm appearing over rolandic cortex. Although SMR biofeedback training has been successfully applied to various forms of epilepsy in humans, its potential use in decreasing hyperactivity has been limited to a few cases in which a seizure history was also a significant feature. The present study represents a first attempt to explore the technique's applicability to the problem of hyperkinesis independent of the epilepsy issue. The results of several months of EEG biofeedback training in a hyperkinetic child tend to corroborate and extend previous findings. Feedback presentations for SMR were contingent on the production of 12–14-Hz activity in the absence of 4–7-Hz slow-wave activity. A substantial increase in SMR occurred with progressive SMR training and was associated with enhanced motor inhibition, as gauged by laboratory measures of muscular tone(chin EMG) and by a global behavioral assessment in the classroom. Opposite trends in motor inhibition occurred when the training procedure was reversed and feedback presentations were contingent on the production of 4–7 Hz in the absence of 12–14-Hz activity. Although the preliminary nature of these results is stressed, the subject population has recently been increased to establish the validity and generality of the findings and will include the use of SMR biofeedback training after medication has been withdrawn.This research was a segment of the junior author's dissertation research.     

Selective Sensation Based Brain-Computer Interface via Mechanical Vibrotactile Stimulation

In this work, mechanical vibrotactile stimulation was applied to subjects’ left and right wrist skins with equal intensity, and a selective sensation perception task was performed to achieve two types of selections similar to motor imagery Brain-Computer Interface. The proposed system was based on event-related desynchronization/synchronization (ERD/ERS), which had a correlation with processing of afferent inflow in human somatosensory system, and attentional effect which modulated the ERD/ERS. The experiments were carried out on nine subjects (without experience in selective sensation), and six of them showed a discrimination accuracy above 80%, three of them above 95%. Comparative experiments with motor imagery (with and without presence of stimulation) were also carried out, which further showed the feasibility of selective sensation as an alternative BCI task complementary to motor imagery. Specifically there was significant improvement () from near 65% in motor imagery (with and without presence of stimulation) to above 80% in selective sensation on some subjects. The proposed BCI modality might well cooperate with existing BCI modalities in the literature in enlarging the widespread usage of BCI system.     

Cerebral Activations Related to Audition-Driven Performance Imagery in Professional Musicians

Functional Magnetic Resonance Imaging (fMRI) was used to study the activation of cerebral motor networks during auditory perception of music in professional keyboard musicians (n = 12). The activation paradigm implied that subjects listened to two-part polyphonic music, while either critically appraising the performance or imagining they were performing themselves. Two-part polyphonic audition and bimanual motor imagery circumvented a hemisphere bias associated with the convention of playing the melody with the right hand. Both tasks activated ventral premotor and auditory cortices, bilaterally, and the right anterior parietal cortex, when contrasted to 12 musically unskilled controls. Although left ventral premotor activation was increased during imagery (compared to judgment), bilateral dorsal premotor and right posterior-superior parietal activations were quite unique to motor imagery. The latter suggests that musicians not only recruited their manual motor repertoire but also performed a spatial transformation from the vertically perceived pitch axis (high and low sound) to the horizontal axis of the keyboard. Imagery-specific activations in controls were seen in left dorsal parietal-premotor and supplementary motor cortices. Although these activations were less strong compared to musicians, this overlapping distribution indicated the recruitment of a general ‘mirror-neuron’ circuitry. These two levels of sensori-motor transformations point towards common principles by which the brain organizes audition-driven music performance and visually guided task performance.     

Knowing Each Random Error of Our Ways,but Hardly Correcting for It: An Instance of Optimal Performance

Random errors are omnipresent in sensorimotor tasks due to perceptual and motor noise. The question is, are humans aware of their random errors on an instance-by-instance basis? The appealing answer would be ‘no’ because it seems intuitive that humans would otherwise immediately correct for the errors online, thereby increasing sensorimotor precision. However, here we show the opposite. Participants pointed to visual targets with varying degree of feedback. After movement completion participants indicated whether they believed they landed left or right of target. Surprisingly, participants'' left/right-discriminability was well above chance, even without visual feedback. Only when forced to correct for the error after movement completion did participants loose knowledge about the remaining error, indicating that random errors can only be accessed offline. When correcting, participants applied the optimal correction gain, a weighting factor between perceptual and motor noise, minimizing end-point variance. Together these results show that humans optimally combine direct information about sensorimotor noise in the system (the current random error), with indirect knowledge about the variance of the perceptual and motor noise distributions. Yet, they only appear to do so offline after movement completion, not while the movement is still in progress, suggesting that during movement proprioceptive information is less precise.     

Rehabilitation potential of post-stroke patients training for kinesthetic movement imagination: Motor and cognitive aspects

The rehabilitation potential of post-stroke patients was evaluated after a rehabilitation procedure using a hand exoskeleton controlled via a brain–computer interface (BCI). Examples are given for parameters describing the motor and cognitive functions and the capacity for kinesthetic movement imagination. It is emphasized that instrumental quantitative methods are important to use for adequate assessment of both the rehabilitation potential and the effectiveness of the BCI + exoskeleton procedure.     

Moving in an Environment of Induced Sensorimotor Incongruence Does Not Influence Pain Sensitivity in Healthy Volunteers: A Randomised Within-Subject Experiment

Objectives

It has been proposed that in the same way that conflict between vestibular and visual inputs leads to motion sickness, conflict between motor commands and sensory information associated with these commands may contribute to some chronic pain states. Attempts to test this hypothesis by artificially inducing a state of sensorimotor incongruence and assessing self-reported pain have yielded equivocal results. To help clarify the effect sensorimotor incongruence has on pain we investigated the effect of moving in an environment of induced incongruence on pressure pain thresholds (PPT) and the pain experienced immediately on completion of PPT testing.

Methods

Thirty-five healthy subjects performed synchronous and asynchronous upper-limb movements with and without mirror visual feedback in random order. We measured PPT over the elbow and the pain evoked by testing. Generalised linear mixed-models were performed for each outcome. Condition (four levels) and baseline values for each outcome were within-subject factors.

Results

There was no effect of condition on PPT (p = 0.887) or pressure-evoked pain (p = 0.771). A sensitivity analysis using only the first PPT measure after each condition confirmed the result (p = 0.867).

Discussion

Inducing a state of movement related sensorimotor incongruence in the upper-limb of healthy volunteers does not influence PPT, nor the pain evoked by testing. We found no evidence that sensorimotor incongruence upregulates the nociceptive system in healthy volunteers.     

Continuous three-dimensional control of a virtual helicopter using a motor imagery based brain-computer interface

Brain-computer interfaces (BCIs) allow a user to interact with a computer system using thought. However, only recently have devices capable of providing sophisticated multi-dimensional control been achieved non-invasively. A major goal for non-invasive BCI systems has been to provide continuous, intuitive, and accurate control, while retaining a high level of user autonomy. By employing electroencephalography (EEG) to record and decode sensorimotor rhythms (SMRs) induced from motor imaginations, a consistent, user-specific control signal may be characterized. Utilizing a novel method of interactive and continuous control, we trained three normal subjects to modulate their SMRs to achieve three-dimensional movement of a virtual helicopter that is fast, accurate, and continuous. In this system, the virtual helicopter's forward-backward translation and elevation controls were actuated through the modulation of sensorimotor rhythms that were converted to forces applied to the virtual helicopter at every simulation time step, and the helicopter's angle of left or right rotation was linearly mapped, with higher resolution, from sensorimotor rhythms associated with other motor imaginations. These different resolutions of control allow for interplay between general intent actuation and fine control as is seen in the gross and fine movements of the arm and hand. Subjects controlled the helicopter with the goal of flying through rings (targets) randomly positioned and oriented in a three-dimensional space. The subjects flew through rings continuously, acquiring as many as 11 consecutive rings within a five-minute period. In total, the study group successfully acquired over 85% of presented targets. These results affirm the effective, three-dimensional control of our motor imagery based BCI system, and suggest its potential applications in biological navigation, neuroprosthetics, and other applications.     

What Do Eye Gaze Metrics Tell Us about Motor Imagery?

Many of the brain structures involved in performing real movements also have increased activity during imagined movements or during motor observation, and this could be the neural substrate underlying the effects of motor imagery in motor learning or motor rehabilitation. In the absence of any objective physiological method of measurement, it is currently impossible to be sure that the patient is indeed performing the task as instructed. Eye gaze recording during a motor imagery task could be a possible way to “spy” on the activity an individual is really engaged in. The aim of the present study was to compare the pattern of eye movement metrics during motor observation, visual and kinesthetic motor imagery (VI, KI), target fixation, and mental calculation. Twenty-two healthy subjects (16 females and 6 males), were required to perform tests in five conditions using imagery in the Box and Block Test tasks following the procedure described by Liepert et al. Eye movements were analysed by a non-invasive oculometric measure (SMI RED250 system). Two parameters describing gaze pattern were calculated: the index of ocular mobility (saccade duration over saccade + fixation duration) and the number of midline crossings (i.e. the number of times the subjects gaze crossed the midline of the screen when performing the different tasks). Both parameters were significantly different between visual imagery and kinesthesic imagery, visual imagery and mental calculation, and visual imagery and target fixation. For the first time we were able to show that eye movement patterns are different during VI and KI tasks. Our results suggest gaze metric parameters could be used as an objective unobtrusive approach to assess engagement in a motor imagery task. Further studies should define how oculomotor parameters could be used as an indicator of the rehabilitation task a patient is engaged in.     

High Theta and Low Alpha Powers May Be Indicative of BCI-Illiteracy in Motor Imagery

In most brain computer interface (BCI) systems, some target users have significant difficulty in using BCI systems. Such target users are called ‘BCI-illiterate’. This phenomenon has been poorly investigated, and a clear understanding of the BCI-illiteracy mechanism or a solution to this problem has not been reported to date. In this study, we sought to demonstrate the neurophysiological differences between two groups (literate, illiterate) with a total of 52 subjects. We investigated recordings under non-task related state (NTS) which is collected during subject is relaxed with eyes open. We found that high theta and low alpha waves were noticeable in the BCI-illiterate relative to the BCI-literate people. Furthermore, these high theta and low alpha wave patterns were preserved across different mental states, such as NTS, resting before motor imagery (MI), and MI states, even though the spatial distribution of both BCI-illiterate and BCI-literate groups did not differ. From these findings, an effective strategy for pre-screening subjects for BCI illiteracy has been determined, and a performance factor that reflects potential user performance has been proposed using a simple combination of band powers. Our proposed performance factor gave an r = 0.59 (r² = 0.34) in a correlation analysis with BCI performance and yielded as much as r = 0.70 (r² = 0.50) when seven outliers were rejected during the evaluation of whole data (N = 61), including BCI competition datasets (N = 9). These findings may be directly applicable to online BCI systems.     

Exploring virtual environments with an EEG-based BCI through motor imagery.

In this paper, we describe the possibility of navigating in a virtual environment using the output signal of an EEG-based Brain-Computer Interface (BCI). The graphical capabilities of virtual reality (VR) should help to create new BCI-paradigms and improve feedback presentation. The objective of this combination is to enhance the subject's learning process of gaining control of the BCI. In this study, the participant had to imagine left or right hand movements while exploring a virtual conference room. By imaging a left hand movement the subject turned virtually to the left inside the room and with right hand imagery to the right. In fact, three trained subjects reached 80% to 100% BCI classification accuracy in the course of the experimental sessions. All subjects were able to achieve a rotation in the VR to the left or right by approximately 45 degrees during one trial.     

Real-Time Subject-Independent Pattern Classification of Overt and Covert Movements from fNIRS Signals

Recently, studies have reported the use of Near Infrared Spectroscopy (NIRS) for developing Brain–Computer Interface (BCI) by applying online pattern classification of brain states from subject-specific fNIRS signals. The purpose of the present study was to develop and test a real-time method for subject-specific and subject-independent classification of multi-channel fNIRS signals using support-vector machines (SVM), so as to determine its feasibility as an online neurofeedback system. Towards this goal, we used left versus right hand movement execution and movement imagery as study paradigms in a series of experiments. In the first two experiments, activations in the motor cortex during movement execution and movement imagery were used to develop subject-dependent models that obtained high classification accuracies thereby indicating the robustness of our classification method. In the third experiment, a generalized classifier-model was developed from the first two experimental data, which was then applied for subject-independent neurofeedback training. Application of this method in new participants showed mean classification accuracy of 63% for movement imagery tasks and 80% for movement execution tasks. These results, and their corresponding offline analysis reported in this study demonstrate that SVM based real-time subject-independent classification of fNIRS signals is feasible. This method has important applications in the field of hemodynamic BCIs, and neuro-rehabilitation where patients can be trained to learn spatio-temporal patterns of healthy brain activity.     

Hybrid EEG-fNIRS Asynchronous Brain-Computer Interface for Multiple Motor Tasks

Non-invasive Brain-Computer Interfaces (BCI) have demonstrated great promise for neuroprosthetics and assistive devices. Here we aim to investigate methods to combine Electroencephalography (EEG) and functional Near-Infrared Spectroscopy (fNIRS) in an asynchronous Sensory Motor rhythm (SMR)-based BCI. We attempted to classify 4 different executed movements, namely, Right-Arm—Left-Arm—Right-Hand—Left-Hand tasks. Previous studies demonstrated the benefit of EEG-fNIRS combination. However, since normally fNIRS hemodynamic response shows a long delay, we investigated new features, involving slope indicators, in order to immediately detect changes in the signals. Moreover, Common Spatial Patterns (CSPs) have been applied to both EEG and fNIRS signals. 15 healthy subjects took part in the experiments and since 25 trials per class were available, CSPs have been regularized with information from the entire population of participants and optimized using genetic algorithms. The different features have been compared in terms of performance and the dynamic accuracy over trials shows that the introduced methods diminish the fNIRS delay in the detection of changes.     

Infants’ Somatotopic Neural Responses to Seeing Human Actions: I’ve Got You under My Skin

Human infants rapidly learn new skills and customs via imitation, but the neural linkages between action perception and production are not well understood. Neuroscience studies in adults suggest that a key component of imitation–identifying the corresponding body part used in the acts of self and other–has an organized neural signature. In adults, perceiving someone using a specific body part (e.g., hand vs. foot) is associated with activation of the corresponding area of the sensory and/or motor strip in the observer’s brain–a phenomenon called neural somatotopy. Here we examine whether preverbal infants also exhibit somatotopic neural responses during the observation of others’ actions. 14-month-old infants were randomly assigned to watch an adult reach towards and touch an object using either her hand or her foot. The scalp electroencephalogram (EEG) was recorded and event-related changes in the sensorimotor mu rhythm were analyzed. Mu rhythm desynchronization was greater over hand areas of sensorimotor cortex during observation of hand actions and was greater over the foot area for observation of foot actions. This provides the first evidence that infants’ observation of someone else using a particular body part activates the corresponding areas of sensorimotor cortex. We hypothesize that this somatotopic organization in the developing brain supports imitation and cultural learning. The findings connect developmental cognitive neuroscience, adult neuroscience, action representation, and behavioral imitation.     

Electrophysiological correlates of action observation treatment in children with cerebral palsy: A pilot study

Action Observation Treatment (AOT) has been shown to be effective in the functional recovery of several clinical populations. However, little is known about the neural underpinnings of the clinical efficacy of AOT in children with Cerebral Palsy (CP). Using electroencephalography (EEG), we recorded µ rhythm desynchronization as an index of sensorimotor cortex modulation during a passive action observation task before and after AOT. The relationship between sensorimotor modulation and clinical outcomes was also assessed. Eight children with CP entered the present randomized controlled crossover pilot study in which the experimental AOT preceded or followed a control Videogame Observation Treatment (VOT). Results provide further evidence of the clinical efficacy of AOT for improving hand motor function in CP, as assessed with the Assisting Hand Assessment (AHA) and Melbourne Assessment of Unilateral Upper Limb Function Scale (MUUL). The novel finding is that AOT increases µ rhythm desynchronization at scalp locations corresponding to the hand representation areas. This effect is associated to functional improvement assessed with the MUUL. These preliminary findings, although referred to as a small sample, suggest that AOT may affect upper limb motor recovery in children with CP and modulate the activation of sensorimotor areas, offering a potential neurophysiological correlate to support the clinical utility of AOT.