Classification of Motor Imagery EEG Based on Time-Domain and Frequency-Domain Dual-Stream Convolutional Neural Network

Background and objectiveAn important task of the brain-computer interface (BCI) of motor imagery is to extract effective time-domain features, frequency-domain features or time-frequency domain features from the raw electroencephalogram (EEG) signals for classification of motor imagery. However, choosing an appropriate method to combine time domain and frequency domain features to improve the performance of motor imagery recognition is still a research hotspot.MethodsIn order to fully extract and utilize the time-domain and frequency-domain features of EEG in classification tasks, this paper proposed a novel dual-stream convolutional neural network (DCNN), which can use time domain signal and frequency domain signal as the inputs, and the extracted time-domain features and frequency-domain features are fused by linear weighting for classification training. Furthermore, the weight can be learned by the DCNN automatically.ResultsThe experiments based on BCI competition II dataset III and BCI competition IV dataset 2a showed that the model proposed by this study has better performance than other conventional methods. The model used time-frequency signal as the inputs had better performance than the model only used time-domain signals or frequency-domain signals. The accuracy of classification was improved for each subject compared with the models only used one signals as the inputs.ConclusionsFurther analysis shown that the fusion weight of different subject is specifically, adjusting the weight coefficient automatically is helpful to improve the classification accuracy.     

Motor Imagery EEG Spectral-Spatial Feature Optimization Using Dual-Tree Complex Wavelet and Neighbourhood Component Analysis

BackgroundFrequency band optimization improves the performance of common spatial pattern (CSP) in motor imagery (MI) tasks classification because MI-related electroencephalograms (EEGs) are highly frequency specific. Many variants of CSP algorithm divided the EEG into various sub bands and then applied CSP. However, the feature dimension of MI-EEG data increases with addition of frequency sub bands and requires efficient feature selection algorithms. The performance of CSP also depends on filtering techniques.MethodIn this study, we designed a dual tree complex wavelet transform based filter bank to filter the EEG into sub bands, instead of traditional filtering methods, which improved the spatial feature extraction efficiency. Further, after filtering EEG into different sub bands, we extracted spatial features from each sub band using CSP and optimized them by a proposed supervised learning framework based on neighbourhood component analysis (NCA). Subsequently, a support vector machine (SVM) is trained to perform classification.ResultsAn experimental study, conducted on two datasets (BCI Competition IV (Dataset 2b), and BCI competition III (Dataset IIIa)), validated the MI classification effectiveness of the proposed method in comparison with standard algorithms such as CSP, Filter bank CSP (CSP), and Discriminative FBCSP (DFBCSP). The average classification accuracy obtained by the proposed method for BCI Competition IV (Dataset 2b), and BCI Competition III (Dataset IIIa) are 84.02 ± 12.2 and 89.1 ± 7.50, respectively and found significant than that achieved by standard methods.ConclusionAchieved superior results suggest that the proposed algorithm can improve the performance of MI-based Brain-computer interface devices.     

Evaluation of EEG Oscillatory Patterns and Cognitive Process during Simple and Compound Limb Motor Imagery

Motor imagery (MI), sharing similar neural representations to motor execution, is regarded as a window to investigate the cognitive motor processes. However, in comparison to simple limb motor imagery, significantly less work has been reported on brain oscillatory patterns induced by compound limb motor imagery which involves several parts of limbs. This study aims to investigate differences of the electroencephalogram (EEG) patterns as well as cognitive process between simple limb motor imagery and compound limb motor imagery. Ten subjects participated in the experiment involving three tasks of simple limb motor imagery (left hand, right hand, feet) and three tasks of compound limb motor imagery (both hands, left hand combined with right foot, right hand combined with left foot). Simultaneous imagination of different limbs contributes to the activation of larger cortical areas as well as two estimated sources located at corresponding motor areas within beta rhythm. Compared with simple limb motor imagery, compound limb motor imagery presents a network with more effective interactions overlying larger brain regions, additionally shows significantly larger causal flow over sensorimotor areas and larger causal density over both sensorimotor areas and neighboring regions. On the other hand, compound limb motor imagery also shows significantly larger 10–11 Hz alpha desynchronization at occipital areas and central theta synchronization. Furthermore, the phase-locking value (PLV) between central and occipital areas of left/right hand combined with contralateral foot imagery is significantly larger than that of simple limb motor imagery. All these findings imply that there exist apparent intrinsic distinctions of neural mechanism between simple and compound limb motor imagery, which presents a more complex effective connectivity network and may involve a more complex cognitive process during information processing.     

The Efficiency of the Brain-Computer Interfaces Based on Motor Imagery with Tactile and Visual Feedback

In this study we compared tactile and visual feedbacks for the motor imagery-based brain–computer interface (BCI) in five healthy subjects. A vertical green bar from the center of the fixing cross to the edge of the screen was used as visual feedback. Vibration motors that were placed on the forearms of the right and the left hands and on the back of the subject’s neck were used as tactile feedback. A vibration signal was used to confirm the correct classification of the EEG patterns of the motor imagery of right and left hand movements and the rest task. The accuracy of recognition in the classification of the three states (right hand movement, left hand movement, and rest) in the BCI without feedback exceeded the random level (33% for the three states) for all the subjects and was rather high (67.8% ± 13.4% (mean ± standard deviation)). Including the visual and tactile feedback in the BCI did not significantly change the mean accuracy of recognition of mental states for all the subjects (70.5% ± 14.8% for the visual feedback and 65.9% ± 12.4% for the tactile feedback). The analysis of the dynamics of the movement imagery skill in BCI users with the tactile and visual feedback showed no significant differences between these types of feedback. Thus, it has been found that the tactile feedback can be used in the motor imagery-based BCI instead of the commonly used visual feedback, which greatly expands the possibilities of the practical application of the BCI.     

基于EMD的左右手运动想象脑电信号分析研究

基于经验模态分解（EMD）理论,提出一种左右手运动想象脑电信号分析方法。首先利用时间窗对脑电信号数据进行划分,对每段数据通过经验模态分解法将其分解为一组固有模态函数IMF,提取主要信号所在的IMF层去除信号中的噪声。对含有主要信号的几层IMF进行Hilbert变换,得到瞬时频率与对应的瞬时幅值。再提取左右手想象的特定频段mu节律和beta节律的能量信号作为特征,分别利用支持向量机（SVM）和Fisher进行了分类比较。对EMD和小波包在去噪和特征提取进行了比较。结果表明,EMD是一种很有效的去噪方法,经过EMD分解后提取的能量信号在区分左右手想象上更具有优势,识别率高。     

Selective Effect of Physical Fatigue on Motor Imagery Accuracy

While the use of motor imagery (the mental representation of an action without overt execution) during actual training sessions is usually recommended, experimental studies examining the effect of physical fatigue on subsequent motor imagery performance are sparse and yielded divergent findings. Here, we investigated whether physical fatigue occurring during an intense sport training session affected motor imagery ability. Twelve swimmers (nine males, mean age 15.5 years) conducted a 45 min physically-fatiguing protocol where they swam from 70% to 100% of their maximal aerobic speed. We tested motor imagery ability immediately before and after fatigue state. Participants randomly imagined performing a swim turn using internal and external visual imagery. Self-reports ratings, imagery times and electrodermal responses, an index of alertness from the autonomic nervous system, were the dependent variables. Self-reports ratings indicated that participants did not encounter difficulty when performing motor imagery after fatigue. However, motor imagery times were significantly shortened during posttest compared to both pretest and actual turn times, thus indicating reduced timing accuracy. Looking at the selective effect of physical fatigue on external visual imagery did not reveal any difference before and after fatigue, whereas significantly shorter imagined times and electrodermal responses (respectively 15% and 48% decrease, p<0.001) were observed during the posttest for internal visual imagery. A significant correlation (r = 0.64; p<0.05) was observed between motor imagery vividness (estimated through imagery questionnaire) and autonomic responses during motor imagery after fatigue. These data support that unlike local muscle fatigue, physical fatigue occurring during intense sport training sessions is likely to affect motor imagery accuracy. These results might be explained by the updating of the internal representation of the motor sequence, due to temporary feedback originating from actual motor practice under fatigue. These findings provide insights to the co-dependent relationship between mental and motor processes.     

Suppression of F-VEP during isoflurane-induced EEG suppression

We recorded visual evoked potentials (VEPs) to flash stimuli in moderately deep anaesthesia when EEG showed burst suppression pattern. Flash VEPs could consistently be recorded in all 8 test subjects during bursts but not during suppressions. We conclude that during isoflurane-induced EEG suppression VEPs to flash stimuli are also suppressed. This effect should be taken into account in evoked potential testing during anaesthesia.     

Effect of Motor Imagery in Children with Unilateral Cerebral Palsy: fMRI Study

Background

Motor imagery is considered as a promising therapeutic tool for rehabilitation of motor planning problems in patients with cerebral palsy. However motor planning problems may lead to poor motor imagery ability.

Aim

The aim of this functional magnetic resonance imaging study was to examine and compare brain activation following motor imagery tasks in patients with hemiplegic cerebral palsy with left or right early brain lesions. We tested also the influence of the side of imagined hand movement.

Method

Twenty patients with clinical hemiplegic cerebral palsy (sixteen males, mean age 12 years and 10 months, aged 6 years 10 months to 20 years 10 months) participated in this study. Using block design, brain activations following motor imagery of a simple opening-closing hand movement performed by either the paretic or nonparetic hand was examined.

Results

During motor imagery tasks, patients with early right brain damages activated bilateral fronto-parietal network that comprise most of the nodes of the network well described in healthy subjects. Inversely, in patients with left early brain lesion brain activation following motor imagery tasks was reduced, compared to patients with right brain lesions. We found also a weak influence of the side of imagined hand movement.

Conclusion

Decreased activations following motor imagery in patients with right unilateral cerebral palsy highlight the dominance of the left hemisphere during motor imagery tasks. This study gives neuronal substrate to propose motor imagery tasks in unilateral cerebral palsy rehabilitation at least for patients with right brain lesions.     

The Impact of Different Visual Feedbacks in User Training on Motor Imagery Control in BCI

The challenges of research into brain–computer interfaces (BCI) include significant individual differences in learning pace and in the effective operation of BCI devices. The use of neurofeedback training is a popular method of improving the effectiveness BCI operation. The purpose of the present study was to determine to what extent it is possible to improve the effectiveness of operation of sensorimotor rhythm-based brain–computer interfaces (SMR-BCI) by supplementing user training with elements modifying the characteristics of visual feedback. Four experimental groups had training designed to reinforce BCI control by: visual feedback in the form of dummy faces expressing emotions (Group 1); flashing the principal elements of visual feedback (Group 2) and giving both visual feedbacks in one condition (Group 3). The fourth group participated in training with no modifications (Group 4). Training consisted of a series of trials where the subjects directed a ball into a basket located to the right or left side of the screen. In Group 1 a schematic image a face, placed on the controlled object, showed various emotions, depending on the accuracy of control. In Group 2, the cue and targets were flashed with different frequency (4 Hz) than the remaining elements visible on the monitor. Both modifications were also used simultaneously in Group 3. SMR activity during the task was recorded before and after the training. In Group 3 there was a significant improvement in SMR control, compared to subjects in Group 2 and 4 (control). Differences between subjects in Groups 1, 2 and 4 (control) were insignificant. This means that relatively small changes in the training procedure may significantly impact the effectiveness of BCI control. Analysis of behavioural data acquired from all participants at training showed greater effectiveness in directing the object towards the right side of the screen. Subjects with the greatest improvement in SMR control showed a significantly lower difference in the accuracy of rightward and leftward movement than others.     

EEG Topographic Mapping of Visual and Kinesthetic Imagery in Swimmers

This study investigated differences in QEEG measures between kinesthetic and visual imagery of a 100-m swim in 36 elite competitive swimmers. Background information and post-trial checks controlled for the modality of imagery, swimming skill level, preferred imagery style, intensity of image and task equality. Measures of EEG relative magnitude in theta, low (7–9 Hz) and high alpha (8–10 Hz), and low and high beta were taken from 19 scalp sites during baseline, visual, and kinesthetic imagery. QEEG magnitudes in the low alpha band during the visual and kinesthetic conditions were attenuated from baseline in low band alpha but no changes were seen in any other bands. Swimmers produced more low alpha EEG magnitude during visual versus kinesthetic imagery. This was interpreted as the swimmers having a greater efficiency at producing visual imagery. Participants who reported a strong intensity versus a weaker feeling of the image (kinesthetic) had less low alpha magnitude, i.e., there was use of more cortical resources, but not for the visual condition. These data suggest that low band (7–9 Hz) alpha distinguishes imagery modalities from baseline, visual imagery requires less cortical resources than kinesthetic imagery, and that intense feelings of swimming requires more brain activity than less intense feelings.     

Online Classification of Multiple Motor Imagery Tasks Using Filter Bank Based Maximum-a-Posteriori Common Spatial Pattern Filters

ObjectiveThe main objective of this paper is to propose a novel technique, called filter bank maximum a-posteriori common spatial pattern (FB-MAP-CSP) algorithm, for online classification of multiple motor imagery activities using electroencephalography (EEG) signals. The proposed technique addresses the overfitting issue of CSP in addition to utilizing the spectral information of EEG signals inside the framework of filter banks while extending it to more than two conditions.Materials and methodsThe classification of motor imagery signals is based upon the detection of event-related de-synchronization (ERD) phenomena in the μ and β rhythms of EEG signals. Accordingly, two modifications in the existing MAP-CSP technique are presented: (i) The (pre-processed) EEG signals are spectrally filtered by a bank of filters lying in the μ and β brainwave frequency range, (ii) the framework of MAP-CSP is extended to deal with multiple (more than two) motor imagery tasks classification and the spatial filters thus obtained are calculated for each sub-band, separately. Subsequently, the most imperative features over all sub-bands are selected and un-regularized linear discriminant analysis is employed for classification of multiple motor imagery tasks.ResultsPublicly available dataset (BCI Competition IV Dataset I) is used to validate the proposed method i.e. FB-MAP-CSP. The results show that the proposed method yields superior classification results, in addition to be computationally more efficient in the case of online implementation, as compared to the conventional CSP based techniques and its variants for multiclass motor imagery classification.ConclusionThe proposed FB-MAP-CSP algorithm is found to be a potential / superior method for classifying multi-condition motor imagery EEG signals in comparison to FBCSP based techniques.     

基于PSoC的脑电信号采集系统的设计

脑电信号是一种很重要的生物医学信号,它是临床医学诊断和脑科学研究的一种重要手段。本文介绍了基于Cypress PSoC~(TM)可编程片上系统的脑电信号采集系统的整个设计过程,包括硬件组成和软件设计方法。通过PSoC芯片特有的可编程模拟系统和数字系统,可以把大量的外围器件集成到芯片的内部,从而提高了硬件系统的集成度和可靠性;加上功能强大的PSOC Designer集成开发环境,提高了开发效率,而且系统的软硬件升级也更加容易了。     

Mental Representation and Mental Practice: Experimental Investigation on the Functional Links between Motor Memory and Motor Imagery

Recent research on mental representation of complex action has revealed distinct differences in the structure of representational frameworks between experts and novices. More recently, research on the development of mental representation structure has elicited functional changes in novices'' representations as a result of practice. However, research investigating if and how mental practice adds to this adaptation process is lacking. In the present study, we examined the influence of mental practice (i.e., motor imagery rehearsal) on both putting performance and the development of one''s representation of the golf putt during early skill acquisition. Novice golfers (N = 52) practiced the task of golf putting under one of four different practice conditions: mental, physical, mental-physical combined, and no practice. Participants were tested prior to and after a practice phase, as well as after a three day retention interval. Mental representation structures of the putt were measured, using the structural dimensional analysis of mental representation. This method provides psychometric data on the distances and groupings of basic action concepts in long-term memory. Additionally, putting accuracy and putting consistency were measured using two-dimensional error scores of each putt. Findings revealed significant performance improvements over the course of practice together with functional adaptations in mental representation structure. Interestingly, after three days of practice, the mental representations of participants who incorporated mental practice into their practice regime displayed representation structures that were more similar to a functional structure than did participants who did not incorporate mental practice. The findings of the present study suggest that mental practice promotes the cognitive adaptation process during motor learning, leading to more elaborate representations than physical practice only.     

基于主成分分析的音乐事件相关脑电变化研究

目的:对在不同音乐背景下获取的脑电数据进行主成分分析.方法:在受测者闭眼欣赏喜欢音乐,闭眼欣赏不喜欢音乐,安静闭眼期间获得脑电数据,并对其进行主成分分析,提取第一主成分,主成分荷载,得出在不同条件和不同事件下主成分的变化规律以及影响主成分变化脑电极的变化规律.数据处理和统计分析采用独立设计的脑电图分析工具箱和主成分分析程序.结果:脑电图数据经过主成分分析后,第一主成分贡献率变化就有比较明显的规律,在闭眼欣赏喜欢音乐时,第一主成分贡献率明显比欣赏不喜欢音乐和安静闭眼时高.在前头部,欣赏喜欢音乐时和欣赏不喜欢音乐时,影响第一主成分的脑电极分别在右前头部和左前头部脑电信号比较强.结论:在不同音乐背景下,大脑左右前头部会受不同音乐所影响,同时也会对人的感觉,情感,情绪等会产生重要的影响.     

Motor Imagery Cognitive Network after Left Ischemic Stroke: Study of the Patients during Mental Rotation Task

Although motor imagery could improve motor rehabilitation, the detailed neural mechanisms of motor imagery cognitive process of stroke patients, particularly from functional network perspective, remain unclear. This study investigated functional brain network properties in each cognitive sub-stage of motor imagery of stroke patients with ischemic lesion in left hemisphere to reveal the impact of stroke on the cognition of motor imagery. Both stroke patients and control subjects participated in mental rotation task, which includes three cognitive sub-stages: visual stimulus perception, mental rotation and response cognitive process. Event-related electroencephalograph was recorded and interdependence between two different cortical areas was assessed by phase synchronization. Both global and nodal properties of functional networks in three sub-stages were statistically analyzed. Phase synchronization of stroke patients significantly reduced in mental rotation sub-stage. Longer characteristic path length and smaller global clustering coefficient of functional network were observed in patients in mental rotation sub-stage which implied the impaired segregation and integration. Larger nodal clustering coefficient and betweenness in contralesional occipitoparietal and frontal area respectively were observed in patients in all sub-stages. In addition, patients also showed smaller betweenness in ipsilesional central-parietal area in response sub-stage. The compensatory effects on local connectedness and centrality indicated the neuroplasticity in contralesional hemisphere. The functional brain networks of stroke patients demonstrated significant alterations and compensatory effects during motor imagery.     

Human Decision Making Based on Variations in Internal Noise: An EEG Study

Perceptual decision making is prone to errors, especially near threshold. Physiological, behavioural and modeling studies suggest this is due to the intrinsic or ‘internal’ noise in neural systems, which derives from a mixture of bottom-up and top-down sources. We show here that internal noise can form the basis of perceptual decision making when the external signal lacks the required information for the decision. We recorded electroencephalographic (EEG) activity in listeners attempting to discriminate between identical tones. Since the acoustic signal was constant, bottom-up and top-down influences were under experimental control. We found that early cortical responses to the identical stimuli varied in global field power and topography according to the perceptual decision made, and activity preceding stimulus presentation could predict both later activity and behavioural decision. Our results suggest that activity variations induced by internal noise of both sensory and cognitive origin are sufficient to drive discrimination judgments.     

基于关联维数和Lyapunov指数高频脑电非线性动力学研究

目的:探讨利用非线性动力学理论特征值来区分老年人和青年人脑电的差异。方法:采用非线性动力学的关联维数和lyapunov指数对老年组和青年组的安静闭眼、安静睁眼和N-back字母记忆事件等的高频脑电进行特征提取,对特征值进行统计分析,分析两组特征值之间的差别。结果:安静闭眼和安静睁眼事件下,老年组多数导联的关联维数和lyapunov指数的特征值与青年组的特征值存在显著差异(P均0.05);N-back字母记忆事件下,老人多数导联的关联维数特征值大于青年的特征值,但老年组只有14导联的lyapunov指数特征值大于青年组的特征值(P均0.05)。不管有无思维活动,人脑在矢状线上的五个导联Fz、FCz、Cz、CPz、Pz的关联维数和lyapunov指数的特征值都处在峰值处。结论:老年人相比青年人脑功能弱、能量发放少;在思维活动情况下,老年需要更多地发放能量来完成相同的思维活动。安静闭眼和睁眼以及N-back字母记忆事件等三个事件的关联维数和lyapunov指数特征值可用来区分老年组和青年组脑电的差异。     

Motor Imagery Learning Modulates Functional Connectivity of Multiple Brain Systems in Resting State

Background

Learning motor skills involves subsequent modulation of resting-state functional connectivity in the sensory-motor system. This idea was mostly derived from the investigations on motor execution learning which mainly recruits the processing of sensory-motor information. Behavioral evidences demonstrated that motor skills in our daily lives could be learned through imagery procedures. However, it remains unclear whether the modulation of resting-state functional connectivity also exists in the sensory-motor system after motor imagery learning.

Methodology/Principal Findings

We performed a fMRI investigation on motor imagery learning from resting state. Based on previous studies, we identified eight sensory and cognitive resting-state networks (RSNs) corresponding to the brain systems and further explored the functional connectivity of these RSNs through the assessments, connectivity and network strengths before and after the two-week consecutive learning. Two intriguing results were revealed: (1) The sensory RSNs, specifically sensory-motor and lateral visual networks exhibited greater connectivity strengths in precuneus and fusiform gyrus after learning; (2) Decreased network strength induced by learning was proved in the default mode network, a cognitive RSN.

Conclusions/Significance

These results indicated that resting-state functional connectivity could be modulated by motor imagery learning in multiple brain systems, and such modulation displayed in the sensory-motor, visual and default brain systems may be associated with the establishment of motor schema and the regulation of introspective thought. These findings further revealed the neural substrates underlying motor skill learning and potentially provided new insights into the therapeutic benefits of motor imagery learning.     

Implementation of Motor Imagery during Specific Aerobic Training Session in Young Tennis Players

The aim of this study was to investigate the effects of implementing motor imagery (MI) during specific tennis high intensity intermittent training (HIIT) sessions on groundstroke performance in young elite tennis players. Stroke accuracy and ball velocity of forehand and backhand drives were evaluated in ten young tennis players, immediately before and after having randomly performed two HIIT sessions. One session included MI exercises during the recovery phases, while the other included verbal encouragements for physical efforts and served as control condition. Results revealed that similar cardiac demand was observed during both sessions, while implementing MI maintained groundstroke accuracy. Embedding MI during HIIT enabled the development of physical fitness and the preservation of stroke performance. These findings bring new insight to tennis and conditioning coaches in order to fulfil the benefits of specific playing HIIT sessions, and therefore to optimise the training time.     

基于奇异值第一主成分的睡眠脑电分期方法研究

目的:脑电信号含多种噪声和伪迹,信噪比较低,特征提取前必须进行复杂的预处理,严重影响睡眠分期的速度。鉴于此,本文提出一种基于奇异值第一主成分的睡眠脑电分期方法,该方法抗噪性能较强,可省去预处理过程,减少计算量,提高睡眠分期的效率。方法:对未经过预处理的睡眠脑电进行奇异系统分析,研究奇异谱曲线,提取奇异值第一主成分,探索其随睡眠状态变化的规律。并通过支持向量机利用奇异值第一主成分对睡眠分期。结果:奇异值第一主成分不仅能表征脑电信号主体,而且可以抑制噪声、降低维数。随着睡眠的深入,奇异值第一主成分的值逐渐增大,但在REM期处于S1期和S2期之间。经MIT-BIH睡眠数据库中5例同导联位置的脑电数据测试(仅1导脑电数据),睡眠脑电分期的准确率达到86.4%。结论:在未对脑电信号进行预处理的情况下,提取的睡眠脑电的奇异值第一主成分能有效表征睡眠状态,是一种有效的睡眠分期依据。本文运用提出的方法仅采用1导脑电数据,就能得到较为满意的睡眠分期结果。该方法有较强的分类性能,且抗噪能力强,不需要对脑电作复杂的预处理,计算量小,方法简单,很大程度上提高了睡眠分期的效率。