Proprioceptive Feedback and Brain Computer Interface (BCI) Based Neuroprostheses

Brain computer interface (BCI) technology has been proposed for motor neurorehabilitation, motor replacement and assistive technologies. It is an open question whether proprioceptive feedback affects the regulation of brain oscillations and therefore BCI control. We developed a BCI coupled on-line with a robotic hand exoskeleton for flexing and extending the fingers. 24 healthy participants performed five different tasks of closing and opening the hand: (1) motor imagery of the hand movement without any overt movement and without feedback, (2) motor imagery with movement as online feedback (participants see and feel their hand, with the exoskeleton moving according to their brain signals, (3) passive (the orthosis passively opens and closes the hand without imagery) and (4) active (overt) movement of the hand and rest. Performance was defined as the difference in power of the sensorimotor rhythm during motor task and rest and calculated offline for different tasks. Participants were divided in three groups depending on the feedback receiving during task 2 (the other tasks were the same for all participants). Group 1 (n = 9) received contingent positive feedback (participants'' sensorimotor rhythm (SMR) desynchronization was directly linked to hand orthosis movements), group 2 (n = 8) contingent “negative” feedback (participants'' sensorimotor rhythm synchronization was directly linked to hand orthosis movements) and group 3 (n = 7) sham feedback (no link between brain oscillations and orthosis movements). We observed that proprioceptive feedback (feeling and seeing hand movements) improved BCI performance significantly. Furthermore, in the contingent positive group only a significant motor learning effect was observed enhancing SMR desynchronization during motor imagery without feedback in time. Furthermore, we observed a significantly stronger SMR desynchronization in the contingent positive group compared to the other groups during active and passive movements. To summarize, we demonstrated that the use of contingent positive proprioceptive feedback BCI enhanced SMR desynchronization during motor tasks.     

EEG Based Brain Computer Interface for Controlling a Robot Arm Movement Through Thought

Background

The Brain Computer Interfaces (BCI) are devices allowing direct communication between the brain of a user and a machine. This technology can be used by disabled people in order to improve their independence and maximize their capabilities such as finding an object in the environment. Such devices can be realized by the non-invasive measurement of information from the cortex by electroencephalography (EEG).

基于节律性脑电信号的脑-机接口

脑-机接口系统是一个不依靠外周神经和肌肉组织等而实现大脑和外界装置之间直接的交流和控制的通道。它为那些运动障碍的残疾人表达自己的意愿和实现对外部设备的控制提供了一种新的强大的技术支持。基于脑电的脑-机接口作为一种非侵入型的技术引起了该领域很多人的关注。基于脑电的脑-机接口采用了很多种类型的脑电信号。其中,振荡性的脑电图由于有较高的幅值和对噪声不敏感等特性而体现出极大的优势。也是由于这些原因,振荡性的脑电图变成了脑-机接口的应用中非常成功的设计之一。本文要介绍主要的基于脑电的脑-机接口中的两种,分别是稳态视觉诱发电位和基于运动本体感觉节律的脑-机接口。作者将详细的叙述该研究的生理背景、脑-机接口的参数,以及该系统的构造及信号处理的方法,并且会演示一些具有潜在应用价值的科研成果。     

Brain–computer interfaces for dissecting cognitive processes underlying sensorimotor control

Sensorimotor control engages cognitive processes such as prediction, learning, and multisensory integration. Understanding the neural mechanisms underlying these cognitive processes with arm reaching is challenging because we currently record only a fraction of the relevant neurons, the arm has nonlinear dynamics, and multiple modalities of sensory feedback contribute to control. A brain–computer interface (BCI) is a well-defined sensorimotor loop with key simplifying advantages that address each of these challenges, while engaging similar cognitive processes. As a result, BCI is becoming recognized as a powerful tool for basic scientific studies of sensorimotor control. Here, we describe the benefits of BCI for basic scientific inquiries and review recent BCI studies that have uncovered new insights into the neural mechanisms underlying sensorimotor control.     

The Impact of Different Visual Feedbacks in User Training on Motor Imagery Control in BCI

The challenges of research into brain–computer interfaces (BCI) include significant individual differences in learning pace and in the effective operation of BCI devices. The use of neurofeedback training is a popular method of improving the effectiveness BCI operation. The purpose of the present study was to determine to what extent it is possible to improve the effectiveness of operation of sensorimotor rhythm-based brain–computer interfaces (SMR-BCI) by supplementing user training with elements modifying the characteristics of visual feedback. Four experimental groups had training designed to reinforce BCI control by: visual feedback in the form of dummy faces expressing emotions (Group 1); flashing the principal elements of visual feedback (Group 2) and giving both visual feedbacks in one condition (Group 3). The fourth group participated in training with no modifications (Group 4). Training consisted of a series of trials where the subjects directed a ball into a basket located to the right or left side of the screen. In Group 1 a schematic image a face, placed on the controlled object, showed various emotions, depending on the accuracy of control. In Group 2, the cue and targets were flashed with different frequency (4 Hz) than the remaining elements visible on the monitor. Both modifications were also used simultaneously in Group 3. SMR activity during the task was recorded before and after the training. In Group 3 there was a significant improvement in SMR control, compared to subjects in Group 2 and 4 (control). Differences between subjects in Groups 1, 2 and 4 (control) were insignificant. This means that relatively small changes in the training procedure may significantly impact the effectiveness of BCI control. Analysis of behavioural data acquired from all participants at training showed greater effectiveness in directing the object towards the right side of the screen. Subjects with the greatest improvement in SMR control showed a significantly lower difference in the accuracy of rightward and leftward movement than others.     

Improving the Performance of an EEG-Based Motor Imagery Brain Computer Interface Using Task Evoked Changes in Pupil Diameter

For individuals with high degrees of motor disability or locked-in syndrome, it is impractical or impossible to use mechanical switches to interact with electronic devices. Brain computer interfaces (BCIs) can use motor imagery to detect interaction intention from users but lack the accuracy of mechanical switches. Hence, there exists a strong need to improve the accuracy of EEG-based motor imagery BCIs attempting to implement an on/off switch. Here, we investigate how monitoring the pupil diameter of a person as a psycho-physiological parameter in addition to traditional EEG channels can improve the classification accuracy of a switch-like BCI. We have recently noticed in our lab (work not yet published) how motor imagery is associated with increases in pupil diameter when compared to a control rest condition. The pupil diameter parameter is easily accessible through video oculography since most gaze tracking systems report pupil diameter invariant to head position. We performed a user study with 30 participants using a typical EEG based motor imagery BCI. We used common spatial patterns to separate motor imagery, signaling movement intention, from a rest control condition. By monitoring the pupil diameter of the user and using this parameter as an additional feature, we show that the performance of the classifier trying to discriminate motor imagery from a control condition improves over the traditional approach using just EEG derived features. Given the limitations of EEG to construct highly robust and reliable BCIs, we postulate that multi-modal approaches, such as the one presented here that monitor several psycho-physiological parameters, can be a successful strategy in making BCIs more accurate and less vulnerable to constraints such as requirements for long training sessions or high signal to noise ratio of electrode channels.     

Enhancing Sensorimotor Activity by Controlling Virtual Objects with Gaze

This fMRI work studies brain activity of healthy volunteers who manipulated a virtual object in the context of a digital game by applying two different control methods: using their right hand or using their gaze. The results show extended activations in sensorimotor areas, not only when participants played in the traditional way (using their hand) but also when they used their gaze to control the virtual object. Furthermore, with the exception of the primary motor cortex, regional motor activity was similar regardless of what the effector was: the arm or the eye. These results have a potential application in the field of the neurorehabilitation as a new approach to generate activation of the sensorimotor system to support the recovery of the motor functions.     

Large-Scale Assessment of a Fully Automatic Co-Adaptive Motor Imagery-Based Brain Computer Interface

In the last years Brain Computer Interface (BCI) technology has benefited from the development of sophisticated machine leaning methods that let the user operate the BCI after a few trials of calibration. One remarkable example is the recent development of co-adaptive techniques that proved to extend the use of BCIs also to people not able to achieve successful control with the standard BCI procedure. Especially for BCIs based on the modulation of the Sensorimotor Rhythm (SMR) these improvements are essential, since a not negligible percentage of users is unable to operate SMR-BCIs efficiently. In this study we evaluated for the first time a fully automatic co-adaptive BCI system on a large scale. A pool of 168 participants naive to BCIs operated the co-adaptive SMR-BCI in one single session. Different psychological interventions were performed prior the BCI session in order to investigate how motor coordination training and relaxation could influence BCI performance. A neurophysiological indicator based on the Power Spectral Density (PSD) was extracted by the recording of few minutes of resting state brain activity and tested as predictor of BCI performances. Results show that high accuracies in operating the BCI could be reached by the majority of the participants before the end of the session. BCI performances could be significantly predicted by the neurophysiological indicator, consolidating the validity of the model previously developed. Anyway, we still found about 22% of users with performance significantly lower than the threshold of efficient BCI control at the end of the session. Being the inter-subject variability still the major problem of BCI technology, we pointed out crucial issues for those who did not achieve sufficient control. Finally, we propose valid developments to move a step forward to the applicability of the promising co-adaptive methods.     

A New Design of Mental State Classification for Subject Independent BCI Systems

BackgroundBrain Computer Interface (BCI) systems have been widely used to develop sustainable assistive technology for people suffering from neurological impairments. A major limitation of current BCI systems is that they are based on Subject-dependent (SD) concept. The SD based BCI system is time consuming and inconvenient for physical or mental disables people and also not suitable for limited computer resources. In order to overcome these problems, recently subject-independent (SI) based BCI concept has been introduced to identify mental states of motor disabled people but the expected outcome of the SI based BCI has not been achieved yet. Hence this paper intends to present an efficient scheme for SI based BCI system. The goal of this research is to develop a method for classifying mental states which can be used by any user. For attaining this target, this study employs a supervised spatial filtering method with four types of feature extraction methods including Katz Fractal Dimension, Sub band Energy, Log Variance and Root Mean Square (RMS) and finally the obtained features are used as input to Linear Discriminant Analysis (LDA) classification model for identifying mental states for SI BCI system.ResultsThe performance of the proposed design is evaluated in several ways such as considering different time window length; different frequency bands; different number of channels. The mean classification accuracy using Katz feature is 84.35% which is the maximum output compare to other features that outperforms the existing methods.ConclusionsOur proposed design will help to make a new technology for development of real-time SI based BCI systems that can be more supportive for the motor disabled patients.     

A reductionist approach to the analysis of learning in brain–computer interfaces

The complexity and scale of brain–computer interface (BCI) studies limit our ability to investigate how humans learn to use BCI systems. It also limits our capacity to develop adaptive algorithms needed to assist users with their control. Adaptive algorithm development is forced offline and typically uses static data sets. But this is a poor substitute for the online, dynamic environment where algorithms are ultimately deployed and interact with an adapting user. This work evaluates a paradigm that simulates the control problem faced by human subjects when controlling a BCI, but which avoids the many complications associated with full-scale BCI studies. Biological learners can be studied in a reductionist way as they solve BCI-like control problems, and machine learning algorithms can be developed and tested in closed loop with the subjects before being translated to full BCIs. The method is to map 19 joint angles of the hand (representing neural signals) to the position of a 2D cursor which must be piloted to displayed targets (a typical BCI task). An investigation is presented on how closely the joint angle method emulates BCI systems; a novel learning algorithm is evaluated, and a performance difference between genders is discussed.     

Continuity of Visual and Auditory Rhythms Influences Sensorimotor Coordination

People often coordinate their movement with visual and auditory environmental rhythms. Previous research showed better performances when coordinating with auditory compared to visual stimuli, and with bimodal compared to unimodal stimuli. However, these results have been demonstrated with discrete rhythms and it is possible that such effects depend on the continuity of the stimulus rhythms (i.e., whether they are discrete or continuous). The aim of the current study was to investigate the influence of the continuity of visual and auditory rhythms on sensorimotor coordination. We examined the dynamics of synchronized oscillations of a wrist pendulum with auditory and visual rhythms at different frequencies, which were either unimodal or bimodal and discrete or continuous. Specifically, the stimuli used were a light flash, a fading light, a short tone and a frequency-modulated tone. The results demonstrate that the continuity of the stimulus rhythms strongly influences visual and auditory motor coordination. Participants'' movement led continuous stimuli and followed discrete stimuli. Asymmetries between the half-cycles of the movement in term of duration and nonlinearity of the trajectory occurred with slower discrete rhythms. Furthermore, the results show that the differences of performance between visual and auditory modalities depend on the continuity of the stimulus rhythms as indicated by movements closer to the instructed coordination for the auditory modality when coordinating with discrete stimuli. The results also indicate that visual and auditory rhythms are integrated together in order to better coordinate irrespective of their continuity, as indicated by less variable coordination closer to the instructed pattern. Generally, the findings have important implications for understanding how we coordinate our movements with visual and auditory environmental rhythms in everyday life.     

Human posterior parietal cortex flexibly determines reference frames for reaching based on sensory context

Current models of sensorimotor transformations emphasize the dominant role of gaze-centered representations for reach planning in the posterior parietal cortex (PPC). Here we exploit fMRI repetition suppression to test whether the sensory modality of a target determines the reference frame used to define the motor goal in the PPC and premotor cortex. We show that when targets are defined visually, the anterior precuneus selectively encodes the motor goal in gaze-centered coordinates, whereas the parieto-occipital junction, Brodman Area 5 (BA 5), and PMd use a mixed gaze- and body-centered representation. In contrast, when targets are defined by unseen proprioceptive cues, activity in these areas switches to represent the motor goal predominantly in body-centered coordinates. These results support computational models arguing for flexibility in reference frames for action according to sensory context. Critically, they provide neuroanatomical evidence that flexibility is achieved by exploiting a multiplicity of reference frames that can be expressed within individual areas.     

An Electrocorticographic Brain Interface in an Individual with Tetraplegia

Brain-computer interface (BCI) technology aims to help individuals with disability to control assistive devices and reanimate paralyzed limbs. Our study investigated the feasibility of an electrocorticography (ECoG)-based BCI system in an individual with tetraplegia caused by C4 level spinal cord injury. ECoG signals were recorded with a high-density 32-electrode grid over the hand and arm area of the left sensorimotor cortex. The participant was able to voluntarily activate his sensorimotor cortex using attempted movements, with distinct cortical activity patterns for different segments of the upper limb. Using only brain activity, the participant achieved robust control of 3D cursor movement. The ECoG grid was explanted 28 days post-implantation with no adverse effect. This study demonstrates that ECoG signals recorded from the sensorimotor cortex can be used for real-time device control in paralyzed individuals.     

Multifunctional Setup for Studying Human Motor Control Using Transcranial Magnetic Stimulation,Electromyography, Motion Capture,and Virtual Reality

The study of neuromuscular control of movement in humans is accomplished with numerous technologies. Non-invasive methods for investigating neuromuscular function include transcranial magnetic stimulation, electromyography, and three-dimensional motion capture. The advent of readily available and cost-effective virtual reality solutions has expanded the capabilities of researchers in recreating “real-world” environments and movements in a laboratory setting. Naturalistic movement analysis will not only garner a greater understanding of motor control in healthy individuals, but also permit the design of experiments and rehabilitation strategies that target specific motor impairments (e.g. stroke). The combined use of these tools will lead to increasingly deeper understanding of neural mechanisms of motor control. A key requirement when combining these data acquisition systems is fine temporal correspondence between the various data streams. This protocol describes a multifunctional system’s overall connectivity, intersystem signaling, and the temporal synchronization of recorded data. Synchronization of the component systems is primarily accomplished through the use of a customizable circuit, readily made with off the shelf components and minimal electronics assembly skills.     

Prominent Inhibitory Projections Guide Sensorimotor Computation: An Invertebrate Perspective

From single‐cell organisms to complex neural networks, all evolved to provide control solutions to generate context‐ and goal‐specific actions. Neural circuits performing sensorimotor computation to drive navigation employ inhibitory control as a gating mechanism as they hierarchically transform (multi)sensory information into motor actions. Here, the focus is on this literature to critically discuss the proposition that prominent inhibitory projections form sensorimotor circuits. After reviewing the neural circuits of navigation across various invertebrate species, it is argued that with increased neural circuit complexity and the emergence of parallel computations, inhibitory circuits acquire new functions. The contribution of inhibitory neurotransmission for navigation goes beyond shaping the communication that drives motor neurons, and instead includes encoding of emergent sensorimotor representations. A mechanistic understanding of the neural circuits performing sensorimotor computations in invertebrates will unravel the minimum circuit requirements driving adaptive navigation.     

Resting State Networks' Corticotopy: The Dual Intertwined Rings Architecture

How does the brain integrate multiple sources of information to support normal sensorimotor and cognitive functions? To investigate this question we present an overall brain architecture (called “the dual intertwined rings architecture”) that relates the functional specialization of cortical networks to their spatial distribution over the cerebral cortex (or “corticotopy”). Recent results suggest that the resting state networks (RSNs) are organized into two large families: 1) a sensorimotor family that includes visual, somatic, and auditory areas and 2) a large association family that comprises parietal, temporal, and frontal regions and also includes the default mode network. We used two large databases of resting state fMRI data, from which we extracted 32 robust RSNs. We estimated: (1) the RSN functional roles by using a projection of the results on task based networks (TBNs) as referenced in large databases of fMRI activation studies; and (2) relationship of the RSNs with the Brodmann Areas. In both classifications, the 32 RSNs are organized into a remarkable architecture of two intertwined rings per hemisphere and so four rings linked by homotopic connections. The first ring forms a continuous ensemble and includes visual, somatic, and auditory cortices, with interspersed bimodal cortices (auditory-visual, visual-somatic and auditory-somatic, abbreviated as VSA ring). The second ring integrates distant parietal, temporal and frontal regions (PTF ring) through a network of association fiber tracts which closes the ring anatomically and ensures a functional continuity within the ring. The PTF ring relates association cortices specialized in attention, language and working memory, to the networks involved in motivation and biological regulation and rhythms. This “dual intertwined architecture” suggests a dual integrative process: the VSA ring performs fast real-time multimodal integration of sensorimotor information whereas the PTF ring performs multi-temporal integration (i.e., relates past, present, and future representations at different temporal scales).     

Critical decision-speed and information transfer in the "Graz Brain-Computer Interface"

The Graz Brain–Computer Interface (BCI) transforms changes in oscillatory EEG activity into control signals for external devices and feedback. These changes are induced by various motor imageries performed by the user. For this study, 2 different types of motor imagery (movement of the right vs. left hand or both feet) were classified by processing 2 bipolar EEG-channels (derived at electrode positions C3 and C4). After a few sessions, within some weeks, 4 young paraplegic patients learned to control the BCI. In accordance with the participants, decision-speed (trial length) was varied and the information transfer rate (ITR) was calculated for each run. All experimental runs have been feedback-runs employing a simple computer-game-like paradigm. A falling ball had to be led into a randomly marked target halfway down the screen. The horizontal position was controlled by the BCI-output signal and the trial length was varied by the investigator across runs. The goal was to find values for trial length enabling a maximum ITR. Three out of 4 participants had good results after a few runs. Analysis of their last 2 experimental sessions, each containing between 10 and 16 runs, showed that the trial length can be reduced to values around 2 s to obtain the highest possible information transfer. Attainable ITRs were between 5 and 17 bit/min depending on the participant's performance and condition.     

A Co-Adaptive Brain-Computer Interface for End Users with Severe Motor Impairment

Co-adaptive training paradigms for event-related desynchronization (ERD) based brain-computer interfaces (BCI) have proven effective for healthy users. As of yet, it is not clear whether co-adaptive training paradigms can also benefit users with severe motor impairment. The primary goal of our paper was to evaluate a novel cue-guided, co-adaptive BCI training paradigm with severely impaired volunteers. The co-adaptive BCI supports a non-control state, which is an important step toward intuitive, self-paced control. A secondary aim was to have the same participants operate a specifically designed self-paced BCI training paradigm based on the auto-calibrated classifier. The co-adaptive BCI analyzed the electroencephalogram from three bipolar derivations (C3, Cz, and C4) online, while the 22 end users alternately performed right hand movement imagery (MI), left hand MI and relax with eyes open (non-control state). After less than five minutes, the BCI auto-calibrated and proceeded to provide visual feedback for the MI task that could be classified better against the non-control state. The BCI continued to regularly recalibrate. In every calibration step, the system performed trial-based outlier rejection and trained a linear discriminant analysis classifier based on one auto-selected logarithmic band-power feature. In 24 minutes of training, the co-adaptive BCI worked significantly (p = 0.01) better than chance for 18 of 22 end users. The self-paced BCI training paradigm worked significantly (p = 0.01) better than chance in 11 of 20 end users. The presented co-adaptive BCI complements existing approaches in that it supports a non-control state, requires very little setup time, requires no BCI expert and works online based on only two electrodes. The preliminary results from the self-paced BCI paradigm compare favorably to previous studies and the collected data will allow to further improve self-paced BCI systems for disabled users.     

Optimizing the Detection of Wakeful and Sleep-Like States for Future Electrocorticographic Brain Computer Interface Applications

Previous studies suggest stable and robust control of a brain-computer interface (BCI) can be achieved using electrocorticography (ECoG). Translation of this technology from the laboratory to the real world requires additional methods that allow users operate their ECoG-based BCI autonomously. In such an environment, users must be able to perform all tasks currently performed by the experimenter, including manually switching the BCI system on/off. Although a simple task, it can be challenging for target users (e.g., individuals with tetraplegia) due to severe motor disability. In this study, we present an automated and practical strategy to switch a BCI system on or off based on the cognitive state of the user. Using a logistic regression, we built probabilistic models that utilized sub-dural ECoG signals from humans to estimate in pseudo real-time whether a person is awake or in a sleep-like state, and subsequently, whether to turn a BCI system on or off. Furthermore, we constrained these models to identify the optimal anatomical and spectral parameters for delineating states. Other methods exist to differentiate wake and sleep states using ECoG, but none account for practical requirements of BCI application, such as minimizing the size of an ECoG implant and predicting states in real time. Our results demonstrate that, across 4 individuals, wakeful and sleep-like states can be classified with over 80% accuracy (up to 92%) in pseudo real-time using high gamma (70–110 Hz) band limited power from only 5 electrodes (platinum discs with a diameter of 2.3 mm) located above the precentral and posterior superior temporal gyrus.     

Exploring virtual environments with an EEG-based BCI through motor imagery.

In this paper, we describe the possibility of navigating in a virtual environment using the output signal of an EEG-based Brain-Computer Interface (BCI). The graphical capabilities of virtual reality (VR) should help to create new BCI-paradigms and improve feedback presentation. The objective of this combination is to enhance the subject's learning process of gaining control of the BCI. In this study, the participant had to imagine left or right hand movements while exploring a virtual conference room. By imaging a left hand movement the subject turned virtually to the left inside the room and with right hand imagery to the right. In fact, three trained subjects reached 80% to 100% BCI classification accuracy in the course of the experimental sessions. All subjects were able to achieve a rotation in the VR to the left or right by approximately 45 degrees during one trial.