A review on directional information in neural signals for brain-machine interfaces

Brain-machine interfaces (BMIs) can be characterized by the technique used to measure brain activity and by the way different brain signals are translated into commands that control an effector. We give an overview of different approaches and focus on a particular BMI approach: the movement of an artificial effector (e.g. arm prosthesis to the right) by those motor cortical signals that control the equivalent movement of a corresponding body part (e.g. arm movement to the right). This approach has been successfully applied in monkeys and humans by accurately extracting parameters of movements from the spiking activity of multiple single-units. Here, we review recent findings showing that analog neuronal population signals, ranging from intracortical local field potentials over epicortical ECoG to non-invasive EEG and MEG, can also be used to decode movement direction and continuous movement trajectories. Therefore, these signals might provide additional or alternative control for this BMI approach, with possible advantages due to reduced invasiveness.     

Electromyographic activity in the shoulder-neck region according to arm position and glenohumeral torque

The electromyographical (EMG) response to isometric ramp contractions of the right arm, the left arm, and both arms was studied using four pairs of surface electrodes above the right upper trapezius muscle (UT) of six men and six women. Contractions were made against gravity with the active arm(s) in eight positions, ranging from flexion to abduction. To describe arm positions, new, simple terminology was developed. Root mean square (rms)-converted EMG-signals were normalized (EMGnorm) with respect to a reference contraction. The EMGnorm corresponding to a 15 N.m torque in the right glenohumeral (GH) joint was strongly related to the position of the right arm (P less than 0.001). The shape of this relationship depended on the electrode position (P less than 0.001). The ratio between EMGnorm at 30 N.m and 15 N.m GH torques was related to arm position (P less than 0.001) and differed between electrodes (P less than 0.001). A left-side GH torque resulted in right-side (contralateral) EMG activity, typically corresponding to 20%-30% of that obtained during similar right-side GH torque. Bilateral GH torque implied 0%-50% increase in EMG activity as compared to that obtained with the right arm alone. The results have shown that signals from one pair of surface electrodes above UT cannot be taken as representative of the EMG activity from electrodes located elsewhere above UT. The EMG recordings reflected a complex pattern of muscular activation, significantly related to both outwardly visible factors (arm position, GH torque), and within-body servosystems (motor control reflexes).     

A neural network model for reconstructing EMG signals from eight shoulder muscles: consequences for rehabilitation robotics and biofeedback

This paper demonstrates the ability of a fully connected feed forward neural network (FFNN) using the backpropagation training algorithm to predict the electromyography (EMG) signal from eight muscles of the shoulder for both exercises not used for training and EMG signals from subjects not used for training. The network showed a good predictive ability for subjects not used for training (r(2) between 0.33 and 0.84) and for activities not used for training (r(2) between 0.56 and 0.89). This may have applications for patients, physical therapists and doctors to monitor patient performance by reviewing the level of agreement between the patient EMG and the predicted EMG. Coupled with traditional methods of evaluation, EMG can provide an excellent measure of how well a patient has responded or is responding to treatment. Incorporating robotic technology could facilitate the use of EMG as an input to an intelligent decision making algorithm used to increase or decrease the level of difficulty according to patient performance.     

Residual Upper Arm Motor Function Primes Innervation of Paretic Forearm Muscles in Chronic Stroke after Brain-Machine Interface (BMI) Training

Background

Abnormal upper arm-forearm muscle synergies after stroke are poorly understood. We investigated whether upper arm function primes paralyzed forearm muscles in chronic stroke patients after Brain-Machine Interface (BMI)-based rehabilitation. Shaping upper arm-forearm muscle synergies may support individualized motor rehabilitation strategies.

Methods

Thirty-two chronic stroke patients with no active finger extensions were randomly assigned to experimental or sham groups and underwent daily BMI training followed by physiotherapy during four weeks. BMI sessions included desynchronization of ipsilesional brain activity and a robotic orthosis to move the paretic limb (experimental group, n = 16). In the sham group (n = 16) orthosis movements were random. Motor function was evaluated with electromyography (EMG) of forearm extensors, and upper arm and hand Fugl-Meyer assessment (FMA) scores. Patients performed distinct upper arm (e.g., shoulder flexion) and hand movements (finger extensions). Forearm EMG activity significantly higher during upper arm movements as compared to finger extensions was considered facilitation of forearm EMG activity. Intraclass correlation coefficient (ICC) was used to test inter-session reliability of facilitation of forearm EMG activity.

Results

Facilitation of forearm EMG activity ICC ranges from 0.52 to 0.83, indicating fair to high reliability before intervention in both limbs. Facilitation of forearm muscles is higher in the paretic as compared to the healthy limb (p<0.001). Upper arm FMA scores predict facilitation of forearm muscles after intervention in both groups (significant correlations ranged from R = 0.752, p = 0.002 to R = 0.779, p = 0.001), but only in the experimental group upper arm FMA scores predict changes in facilitation of forearm muscles after intervention (R = 0.709, p = 0.002; R = 0.827, p<0.001).

Conclusions

Residual upper arm motor function primes recruitment of paralyzed forearm muscles in chronic stroke patients and predicts changes in their recruitment after BMI training. This study suggests that changes in upper arm-forearm synergies contribute to stroke motor recovery, and provides candidacy guidelines for similar BMI-based clinical practice.     

Multivariate and multiorgan analysis of cardiorespiratory variability signals: the CAP sleep case.

Signals from different systems are analyzed during sleep on a beat-to-beat basis to provide a quantitative measure of synchronization with the heart rate variability (HRV) signal, oscillations of which reflect the action of the autonomic nervous system. Beat-to-beat variability signals synchronized to QRS occurrence on ECG signals were extracted from respiration, electroencephalogram (EEG) and electromyogram (EMG) traces. The analysis was restricted to sleep stage 2. Cyclic alternating pattern (CAP) periods were detected from EEG signals and the following conditions were identified: stage 2 non-CAP (2 NCAP), stage 2 CAP (2 CAP) and stage 2 CAP with myoclonus (2 CAP MC). The coupling relationships between pairs of variability signals were studied in both the time and frequency domains. Passing from 2 NCAP to 2 CAP, sympathetic activation is indicated by tachycardia and reduced respiratory arrhythmia in the heart rate signal. At the same time, we observed a marked link between EEG and HRV at the CAP frequency. During 2 CAP MC, the increased synchronization involved myoclonus and respiration. The underlying mechanism seems to be related to a global control system at the central level that involves the different systems.     

Review on electromyography signal acquisition and processing

Electromyography (EMG) is a technique for recording biomedical electrical signals obtained from the neuromuscular activities. These signals are used to monitor medical abnormalities and activation levels, and also to analyze the biomechanics of any animal movements. In this article, we provide a short review of EMG signal acquisition and processing techniques. The average efficiency of capture of EMG signals with current technologies is around 70%. Once the signal is captured, signal processing algorithms then determine the recognition accuracy, with which signals are decoded for their corresponding purpose (e.g., moving robotic arm, speech recognition, gait analysis). The recognition accuracy can go as high as 99.8%. The accuracy with which the EMG signal is decoded has already crossed 99%, and with improvements in deep learning technology, there is a large scope for improvement in the design hardware that can efficiently capture EMG signals.     

Bispectrum-based features classification for myoelectric control

Surface electromyographic signals provide useful information about motion intentionality. Therefore, they are a suitable reference signal for control purposes. A continuous classification scheme of five upper limb movements applied to a myoelectric control of a robotic arm is presented. This classification is based on features extracted from the bispectrum of four EMG signal channels. Among several bispectrum estimators, this paper is focused on arithmetic mean, median, and trimmed mean estimators, and their ensemble average versions. All bispectrum estimators have been evaluated in terms of accuracy, robustness against outliers, and computational time. The median bispectrum estimator shows low variance and high robustness properties. Two feature reduction methods for the complex bispectrum matrix are proposed. The first one estimates the three classic means (arithmetic, harmonic, and geometric means) from the module of the bispectrum matrix, and the second one estimates the same three means from the square of the real part of the bispectrum matrix. A two-layer feedforward network for movement's classification and a dedicated system to achieve the myoelectric control of a robotic arm were used. It was found that the classification performance in real-time is similar to those obtained off-line by other authors, and that all volunteers in the practical application successfully completed the control task.     

Estimation of dynamic joint torques and trajectory formation from surface electromyography signals using a neural network model

In this study, human arm movement was re-constructed from electromyography (EMG) signals using a forward dynamics model acquired by an artificial neural network within a modular architecture. Dynamic joint torques at the elbow and shoulder were estimated for movements in the horizontal plane from the surface EMG signals of 10 flexor and extensor muscles. Using only the initial conditions of the arm and the EMG time course as input, the network reliably reconstructed a variety of movement trajectories. The results demonstrate that posture maintenance and multijoint movements, entailing complex via-point specification and co-contraction of muscles, can be accurately computed from multiple surface EMG signals. In addition to the model's empirical uses, such as calculation of arm stiffness during motion, it allows evaluation of hypothesized computational mechanisms of the central nervous system such as virtual trajectory control and optimal trajectory planning.     

Persistence of improvements in postural strategies following motor control training in people with recurrent low back pain.

This study investigated long-term effects of training on postural control using the model of deficits in activation of transversus abdominis (TrA) in people with recurrent low back pain (LBP). Nine volunteers with LBP attended four sessions for assessment and/or training (initial, two weeks, four weeks and six months). Training of repeated isolated voluntary TrA contractions were performed at the initial and two-week session with feedback from real-time ultrasound imaging. Home program involved training twice daily for four weeks. Electromyographic activity (EMG) of trunk and deltoid muscles was recorded with surface and fine-wire electrodes. Rapid arm movement and walking were performed at each session, and immediately after training on the first two sessions. Onset of trunk muscle activation relative to prime mover deltoid during arm movements, and the coefficient of variation (CV) of EMG during averaged gait cycle were calculated. Over four weeks of training, onset of TrA EMG was earlier during arm movements and CV of TrA EMG was reduced (consistent with more sustained EMG activity). Changes were retained at six months follow-up (p<0.05). These results show persistence of motor control changes following training and demonstrate that this training approach leads to motor learning of automatic postural control strategies.     

Time-Dependent Statistical and Correlation Properties of Neural Signals during Handwriting

To elucidate the cortical control of handwriting, we examined time-dependent statistical and correlational properties of simultaneously recorded 64-channel electroencephalograms (EEGs) and electromyograms (EMGs) of intrinsic hand muscles. We introduced a statistical method, which offered advantages compared to conventional coherence methods. In contrast to coherence methods, which operate in the frequency domain, our method enabled us to study the functional association between different neural regions in the time domain. In our experiments, subjects performed about 400 stereotypical trials during which they wrote a single character. These trials provided time-dependent EMG and EEG data capturing different handwriting epochs. The set of trials was treated as a statistical ensemble, and time-dependent correlation functions between neural signals were computed by averaging over that ensemble. We found that trial-to-trial variability of both the EMGs and EEGs was well described by a log-normal distribution with time-dependent parameters, which was clearly distinguished from the normal (Gaussian) distribution. We found strong and long-lasting EMG/EMG correlations, whereas EEG/EEG correlations, which were also quite strong, were short-lived with a characteristic correlation durations on the order of 100 ms or less. Our computations of correlation functions were restricted to the spectral range (13–30 Hz) of EEG signals where we found the strongest effects related to handwriting. Although, all subjects involved in our experiments were right-hand writers, we observed a clear symmetry between left and right motor areas: inter-channel correlations were strong if both channels were located over the left or right hemispheres, and 2–3 times weaker if the EEG channels were located over different hemispheres. Although we observed synchronized changes in the mean energies of EEG and EMG signals, we found that EEG/EMG correlations were much weaker than EEG/EEG and EMG/EMG correlations. The absence of strong correlations between EMG and EEG signals indicates that (i) a large fraction of the EEG signal includes electrical activity unrelated to low-level motor variability; (ii) neural processing of cortically-derived signals by spinal circuitry may reduce the correlation between EEG and EMG signals.     

Novel whole body plethysmography system for the continuous characterization of sleep and breathing in a mouse

Sleep is associated with marked alterations in ventilatory control that lead to perturbations in respiratory timing, breathing pattern, ventilation, pharyngeal collapsibility, and sleep-related breathing disorders (SRBD). Mouse models offer powerful insight into the pathogenesis of SRBD; however, methods for obtaining the full complement of continuous, high-fidelity respiratory, electroencephalographic (EEG), and electromyographic (EMG) signals in unrestrained mice during sleep and wake have not been developed. We adapted whole body plethysmography to record EEG, EMG, and respiratory signals continuously in unrestrained, unanesthetized mice. Whole body plethysmography tidal volume and airflow signals and a novel noninvasive surrogate for respiratory effort (respiratory movement signal) were validated against simultaneously measured gold standard signals. Compared with the gold standard, we validated 1) tidal volume (correlation, R(2) = 0.87, P < 0.001; and agreement within 1%, P < 0.001); 2) inspiratory airflow (correlation, R(2) = 0.92, P < 0.001; agreement within 4%, P < 0.001); 3) expiratory airflow (correlation, R(2) = 0.83, P < 0.001); and 4) respiratory movement signal (correlation, R(2) = 0.79-0.84, P < 0.001). The expiratory airflow signal, however, demonstrated a decrease in amplitude compared with the gold standard. Integrating respiratory and EEG/EMG signals, we fully characterized sleep and breathing patterns in conscious, unrestrained mice and demonstrated inspiratory flow limitation in a New Zealand Obese mouse. Our approach will facilitate studies of SRBD mechanisms in inbred mouse strains and offer a powerful platform to investigate the effects of environmental and pharmacological exposures on breathing disturbances during sleep and wakefulness.     

Electromyographic response is altered during robotic surgical training with augmented feedback

There is a growing prevalence of robotic systems for surgical laparoscopy. We previously developed quantitative measures to assess robotic surgical proficiency, and used augmented feedback to enhance training to reduce applied grip force and increase speed. However, there is also a need to understand the physiological demands of the surgeon during robotic surgery, and if training can reduce these demands. Therefore, the goal of this study was to use clinical biomechanical techniques via electromyography (EMG) to investigate the effects of real-time augmented visual feedback during short-term training on muscular activation and fatigue. Twenty novices were trained in three inanimate surgical tasks with the da Vinci Surgical System. Subjects were divided into five feedback groups (speed, relative phase, grip force, video, and control). Time- and frequency-domain EMG measures were obtained before and after training. Surgical training decreased muscle work as found from mean EMG and EMG envelopes. Grip force feedback further reduced average and total muscle work, while speed feedback increased average muscle work and decreased total muscle work. Training also increased the median frequency response as a result of increased speed and/or reduced fatigue during each task. More diverse motor units were recruited as revealed by increases in the frequency bandwidth post-training. We demonstrated that clinical biomechanics using EMG analysis can help to better understand the effects of training for robotic surgery. Real-time augmented feedback during training can further reduce physiological demands. Future studies will investigate other means of feedback such as biofeedback of EMG during robotic surgery training.     

A neural network model for the intersensory coordination involved in goal-directed movements

A neural network model for a sensorimotor system, which was developed to simulate oriented movements in man, is presented. It is composed of a formal neural network comprising two layers: a sensory layer receiving and processing sensory inputs, and a motor layer driving a simulated arm. The sensory layer is an extension of the topological network previously proposed by Kohonen (1984). Two kinds of sensory modality, proprioceptive and exteroceptive, are used to define the arm position. Each sensory cell receives proprioceptive inputs provided by each arm-joint together with the exteroceptive inputs. This sensory layer is therefore a kind of associative layer which integrates two separate sensory signals relating to movement coding. It is connected to the motor layer by means of adaptive synapses which provide a physical link between a motor activity and its sensory consequences. After a learning period, the spatial map which emerges in the sensory layer clearly depends on the sensory inputs and an associative map of both the arm and the extra-personal space is built up if proprioceptive and exteroceptive signals are processed together. The sensorimotor transformations occuring in the junctions linking the sensory and motor layers are organized in such a manner that the simulated arm becomes able to reach towards and track a target in extra-personal space. Proprioception serves to determine the final arm posture adopted and to correct the ongoing movement in cases where changes in the target location occur. With a view to developing a sensorimotor control system with more realistic salient features, a robotic model was coupled with the formal neural network. This robotic implementation of our model shows the capacity of formal neural networks to control the displacement of mechanical devices.     

Fatigue compensation during FES using surface EMG

Muscle fatigue limits the effectiveness of FES when applied to regain functional movements in spinal cord injured (SCI) individuals. The stimulation intensity must be manually increased to provide more force output to compensate for the decreasing muscle force due to fatigue. An artificial neural network (ANN) system was designed to compensate for muscle fatigue during functional electrical stimulation (FES) by maintaining a constant joint angle. Surface electromyography signals (EMG) from electrically stimulated muscles were used to determine when to increase the stimulation intensity when the muscle’s output started to drop.

In two separate experiments on able-bodied subjects seated in hard back chairs, electrical stimulation was continuously applied to fatigue either the biceps (during elbow flexion) or the quadriceps muscle (during leg extension) while recording the surface EMG. An ANN system was created using processed surface EMG as the input, and a discrete fatigue compensation control signal, indicating when to increase the stimulation current, as the output. In order to provide training examples and test the systems’ performance, the stimulation current amplitude was manually increased to maintain constant joint angles. Manual stimulation amplitude increases were required upon observing a significant decrease in the joint angle. The goal of the ANN system was to generate fatigue compensation control signals in an attempt to maintain a constant joint angle.

On average, the systems could correctly predict 78.5% of the instances at which a stimulation increase was required to maintain the joint angle. The performance of these ANN systems demonstrates the feasibility of using surface EMG feedback in an FES control system.     

Development of a robot arm and workstation for the disabled

A robotic workstation for the severely physically disabled is being developed. The prototype system consists of a commercially available arm mounted in a workstation set up for various manipulative tasks. This system has been tested with eight disabled users in a hospital situation and is to undergo evaluation in the homes of disabled users. Based on experience with this system, a new arm has been designed and will be built into a redesigned workstation.     

Restoration of whole body movement: toward a noninvasive brain-machine interface system

This article highlights recent advances in the design of noninvasive neural interfaces based on the scalp electroencephalogram (EEG). The simplest of physical tasks, such as turning the page to read this article, requires an intense burst of brain activity. It happens in milliseconds and requires little conscious thought. But for amputees and stroke victims with diminished motor-sensory skills, this process can be difficult or impossible. Our team at the University of Maryland, in conjunction with the Johns Hopkins Applied Physics Laboratory (APL) and the University of Maryland School of Medicine, hopes to offer these people newfound mobility and dexterity. In separate research thrusts, were using data gleaned from scalp EEG to develop reliable brainmachine interface (BMI) systems that could soon control modern devices such as prosthetic limbs or powered robotic exoskeletons.     

Robust Brain-computer Interface for virtual Keyboard (RoBIK): Project results

Brain-Computer Interface (BCI) is a technology that translates the brain electrical activity into a command for a device such as a robotic arm, a wheelchair or a spelling device. BCIs have long been described as an assistive technology for severely disabled patients because they completely bypass the need for muscular activity. The clinical reality is however dramatically different and most patients who use BCIs today are doing so as part of constraining clinical trials. To achieve the technological transfer from bench to bedside, BCI must gain ease of use and robustness of both measure (electroencephalography [EEG]) and interface (signal processing and applications). The Robust Brain-computer Interface for virtual Keyboard (RoBIK) project aimed at the development of a BCI system for communication that could be used on a daily basis by patients without the help of a trained team of researchers. To guide further developments clinicians first assessed patients’ needs. The prototype subsequently developed consisted in a 14 felt-pad electrodes EEG headset sampling at 256 Hz by an electronic component capable of transmitting signals wirelessly. The application was a virtual keyboard generating a novel stimulation paradigm to elicit P300 Evoked Related Potentials (ERPs) for communication. Raw EEG signals were treated with OpenViBE open-source software including novel signal processing and stimulation techniques.     

An automatic EEG-based sleep staging system with introducing NAoSP and NAoGP as new metrics for sleep staging systems

Different biological signals are recorded in sleep labs during sleep for the diagnosis and treatment of human sleep problems. Classification of sleep stages with electroencephalography (EEG) is preferred to other biological signals due to its advantages such as providing clinical information, cost-effectiveness, comfort, and ease of use. The evaluation of EEG signals taken during sleep by clinicians is a tiring, time-consuming, and error-prone method. Therefore, it is clinically mandatory to determine sleep stages by using software-supported systems. Like all classification problems, the accuracy rate is used to compare the performance of studies in this domain, but this metric can be accurate when the number of observations is equal in classes. However, since there is not an equal number of observations in sleep stages, this metric is insufficient in the evaluation of such systems. For this purpose, in recent years, Cohen’s kappa coefficient and even the sensitivity of NREM1 have been used for comparing the performance of these systems. Still, none of them examine the system from all dimensions. Therefore, in this study, two new metrics based on the polygon area metric, called the normalized area of sensitivity polygon and normalized area of the general polygon, are proposed for the performance evaluation of sleep staging systems. In addition, a new sleep staging system is introduced using the applications offered by the MATLAB program. The existing systems discussed in the literature were examined with the proposed metrics, and the best systems were compared with the proposed sleep staging system. According to the results, the proposed system excels in comparison with the most advanced machine learning methods. The single-channel method introduced based on the proposed metrics can be used for robust and reliable sleep stage classification from all dimensions required for real-time applications.Electronic supplementary materialThe online version of this article (10.1007/s11571-020-09641-2) contains supplementary material, which is available to authorized users.     

Toward electrocorticographic control of a dexterous upper limb prosthesis: building brain-machine interfaces

One of the most exciting and compelling areas of research and development is building brain machine interfaces (BMIs) for controlling prosthetic limbs. Prosthetic limb technology is advancing rapidly, and the modular prosthetic limb (MPL) of the Johns Hopkins University/ Applied Physics Laboratory (JHU/APL) permits actuation with 17 degrees of freedom in 26 articulating joints. There are many signals from the brain that can be leveraged, including the spiking rates of neurons in the cortex, electrocorticographic (ECoG) signals from the surface of the cortex, and electroencephalographic (EEG) signals from the scalp. Unlike microelectrodes that record spikes, ECoG does not penetrate the cortex and has a higher spatial specificity, signal-to-noise ratio, and bandwidth than EEG signals. We have implemented an ECoG-based system for controlling the MPL in the Johns Hopkins Hospital Epilepsy Monitoring Unit, where patients are implanted with ECoG electrode grids for clinical seizure mapping and asked to perform various recorded finger or grasp movements. We have shown that low-frequency local motor potentials (LMPs) and ECoG power in the high gamma frequency (70,150 Hz) range correlate well with grasping parameters, and they stand out as good candidate features for closed-loop control of the MPL.