Silent communication: toward using brain signals

From the 1980s movie Firefox to the more recent Avatar, popular science fiction has speculated about the possibility of a persons thoughts being read directly from his or her brain. Such braincomputer interfaces (BCIs) might allow people who are paralyzed to communicate with and control their environment, and there might also be applications in military situations wherever silent user-to-user communication is desirable. Previous studies have shown that BCI systems can use brain signals related to movements and movement imagery or attention-based character selection. Although these systems have successfully demonstrated the possibility to control devices using brain function, directly inferring which word a person intends to communicate has been elusive. A BCI using imagined speech might provide such a practical, intuitive device. Toward this goal, our studies to date addressed two scientific questions: (1) Can brain signals accurately characterize different aspects of speech? (2) Is it possible to predict spoken or imagined words or their components using brain signals?     

Towards On-line Adaptation of Neuro-prostheses with Neuronal Evaluation Signals

Many experiments have successfully demonstrated that prosthetic devices for restoring lost body functions can in principle be controlled by brain signals. However, stable long-term application of these devices, required for paralyzed patients, may suffer substantially from on-going signal changes for example adapting neural activities or movements of the electrodes recording brain activity. These changes currently require tedious re-learning procedures which are conducted and supervised under laboratory conditions, hampering the everyday use of such devices. As an efficient alternative to current methods we here propose an on-line adaptation scheme that exploits a hypothetical secondary signal source from brain regions reflecting the user’s affective evaluation of the current neuro- prosthetic’s performance. For demonstrating the feasibility of our idea, we simulate a typical prosthetic setup controlling a virtual robotic arm. Hereby we use the additional, hypothetical evaluation signal to adapt the decoding of the intended arm movement which is subjected to large non-stationarities. Even with weak signals and high noise levels typically encountered in recording brain activities, our simulations show that prosthetic devices can be adapted successfully during everyday usage, requiring no special training procedures. Furthermore, the adaptation is shown to be stable against large changes in neural encoding and/or in the recording itself.     

fNIRS Exhibits Weak Tuning to Hand Movement Direction

Functional near-infrared spectroscopy (fNIRS) has become an established tool to investigate brain function and is, due to its portability and resistance to electromagnetic noise, an interesting modality for brain-machine interfaces (BMIs). BMIs have been successfully realized using the decoding of movement kinematics from intra-cortical recordings in monkey and human. Recently, it has been shown that hemodynamic brain responses as measured by fMRI are modulated by the direction of hand movements. However, quantitative data on the decoding of movement direction from hemodynamic responses is still lacking and it remains unclear whether this can be achieved with fNIRS, which records signals at a lower spatial resolution but with the advantage of being portable. Here, we recorded brain activity with fNIRS above different cortical areas while subjects performed hand movements in two different directions. We found that hemodynamic signals in contralateral sensorimotor areas vary with the direction of movements, though only weakly. Using these signals, movement direction could be inferred on a single-trial basis with an accuracy of ∼65% on average across subjects. The temporal evolution of decoding accuracy resembled that of typical hemodynamic responses observed in motor experiments. Simultaneous recordings with a head tracking system showed that head movements, at least up to some extent, do not influence the decoding of fNIRS signals. Due to the low accuracy, fNIRS is not a viable alternative for BMIs utilizing decoding of movement direction. However, due to its relative resistance to head movements, it is promising for studies investigating brain activity during motor experiments.     

A review on directional information in neural signals for brain-machine interfaces

Brain-machine interfaces (BMIs) can be characterized by the technique used to measure brain activity and by the way different brain signals are translated into commands that control an effector. We give an overview of different approaches and focus on a particular BMI approach: the movement of an artificial effector (e.g. arm prosthesis to the right) by those motor cortical signals that control the equivalent movement of a corresponding body part (e.g. arm movement to the right). This approach has been successfully applied in monkeys and humans by accurately extracting parameters of movements from the spiking activity of multiple single-units. Here, we review recent findings showing that analog neuronal population signals, ranging from intracortical local field potentials over epicortical ECoG to non-invasive EEG and MEG, can also be used to decode movement direction and continuous movement trajectories. Therefore, these signals might provide additional or alternative control for this BMI approach, with possible advantages due to reduced invasiveness.     

Multimodal Movement Prediction - Towards an Individual Assistance of Patients

Assistive devices, like exoskeletons or orthoses, often make use of physiological data that allow the detection or prediction of movement onset. Movement onset can be detected at the executing site, the skeletal muscles, as by means of electromyography. Movement intention can be detected by the analysis of brain activity, recorded by, e.g., electroencephalography, or in the behavior of the subject by, e.g., eye movement analysis. These different approaches can be used depending on the kind of neuromuscular disorder, state of therapy or assistive device. In this work we conducted experiments with healthy subjects while performing self-initiated and self-paced arm movements. While other studies showed that multimodal signal analysis can improve the performance of predictions, we show that a sensible combination of electroencephalographic and electromyographic data can potentially improve the adaptability of assistive technical devices with respect to the individual demands of, e.g., early and late stages in rehabilitation therapy. In earlier stages for patients with weak muscle or motor related brain activity it is important to achieve high positive detection rates to support self-initiated movements. To detect most movement intentions from electroencephalographic or electromyographic data motivates a patient and can enhance her/his progress in rehabilitation. In a later stage for patients with stronger muscle or brain activity, reliable movement prediction is more important to encourage patients to behave more accurately and to invest more effort in the task. Further, the false detection rate needs to be reduced. We propose that both types of physiological data can be used in an and combination, where both signals must be detected to drive a movement. By this approach the behavior of the patient during later therapy can be controlled better and false positive detections, which can be very annoying for patients who are further advanced in rehabilitation, can be avoided.     

Artificial feedback for invasive brain–computer interfaces

During the last two decades, considerable progress has been made in the studies of brain–computer interfaces (BCIs)—devices in which motor signals from the brain are registered by multi-electrode arrays and transformed into commands for artificial actuators such as cursors and robotic devices. This review is focused on one problem. Voluntary motor control is based on neurophysiological processes, which strongly depend on the afferent innervation of skin, muscles, and joints. Thus, invasive BCI has to be based on a bidirectional system in which motor control signals are registered by multichannel microelectrodes implanted in motor areas, whereas tactile, proprioceptive, and other useful signals are transported back to the brain through spatiotemporal patterns of intracortical microstimulation (ICMS) delivered to sensory areas. In general, the studies of invasive BCIs have advanced in several directions. The progress of BCIs with artificial sensory feedback will not only help patients, but will also expand base knowledge in the field of human cortical functions.     

深部脑刺激作用机制的探讨

深部脑刺激器（deep brain stimulator）,也经常被称为脑起搏器,是可植入人体设备,并连续不断地传送刺激脉冲到深部脑组织的特定区域,即所谓的深部脑刺激（deep brain stimulation,DBS）.迄今为止,深部脑刺激是治疗严重顽固抗药性运动障碍疾病（如帕金森病,原发性震颤及肌张力异常等）的最有效的外科治疗手段之一.此外,广大的科研工作者也不断地探索应用DBS治疗其他神经及精神异常（如,癫痫和强迫症）的新的临床应用.尽管应用DBS治疗运动障碍非常有效,并也迅速被探索性地应用到其他神经障碍治疗中,但其作用机制仍然不是十分清楚,成为学者们争论的热点.DBS治疗效果的作用机制通常有两种基本的观点：高频刺激抑制学说及高频刺激兴奋学说.基于最近发表的关于中枢神经系统内的高频刺激效应的资料、数据及相关评论,两种机制共存并发挥作用的DBS作用假说被提出,认为DBS通过施加高频刺激干扰并控制了核团病理性紊乱随机活动,同时施加兴奋性刺激到其他基底节的网络,以实现对帕金森病的治疗.     

An Electrocorticographic Brain Interface in an Individual with Tetraplegia

Brain-computer interface (BCI) technology aims to help individuals with disability to control assistive devices and reanimate paralyzed limbs. Our study investigated the feasibility of an electrocorticography (ECoG)-based BCI system in an individual with tetraplegia caused by C4 level spinal cord injury. ECoG signals were recorded with a high-density 32-electrode grid over the hand and arm area of the left sensorimotor cortex. The participant was able to voluntarily activate his sensorimotor cortex using attempted movements, with distinct cortical activity patterns for different segments of the upper limb. Using only brain activity, the participant achieved robust control of 3D cursor movement. The ECoG grid was explanted 28 days post-implantation with no adverse effect. This study demonstrates that ECoG signals recorded from the sensorimotor cortex can be used for real-time device control in paralyzed individuals.     

Shaping the dynamics of a bidirectional neural interface

Progress in decoding neural signals has enabled the development of interfaces that translate cortical brain activities into commands for operating robotic arms and other devices. The electrical stimulation of sensory areas provides a means to create artificial sensory information about the state of a device. Taken together, neural activity recording and microstimulation techniques allow us to embed a portion of the central nervous system within a closed-loop system, whose behavior emerges from the combined dynamical properties of its neural and artificial components. In this study we asked if it is possible to concurrently regulate this bidirectional brain-machine interaction so as to shape a desired dynamical behavior of the combined system. To this end, we followed a well-known biological pathway. In vertebrates, the communications between brain and limb mechanics are mediated by the spinal cord, which combines brain instructions with sensory information and organizes coordinated patterns of muscle forces driving the limbs along dynamically stable trajectories. We report the creation and testing of the first neural interface that emulates this sensory-motor interaction. The interface organizes a bidirectional communication between sensory and motor areas of the brain of anaesthetized rats and an external dynamical object with programmable properties. The system includes (a) a motor interface decoding signals from a motor cortical area, and (b) a sensory interface encoding the state of the external object into electrical stimuli to a somatosensory area. The interactions between brain activities and the state of the external object generate a family of trajectories converging upon a selected equilibrium point from arbitrary starting locations. Thus, the bidirectional interface establishes the possibility to specify not only a particular movement trajectory but an entire family of motions, which includes the prescribed reactions to unexpected perturbations.     

Comparing information about arm movement direction in single channels of local and epicortical field potentials from monkey and human motor cortex.

Cortical field potentials have been used for decades in neurophysiological studies to probe spatio-temporal activity patterns of local populations of neurons. Recently, however, interest in these signals was spurred as they were proposed as potential control signals for neuronal motor prostheses, i.e., for devices fit to record and decode brain activity to restore motor functions in paralyzed patients. Little is known, however, about the functional significance of these cortical field potentials. Here we compared information about arm movement direction in two types of movement related cortical field potentials, obtained during a four direction center-out arm reaching paradigm: local field potentials (LFPs) recorded with intracortical micro-electrodes from monkey motor cortex, and epicortical field potentials (EFPs) recorded with macro-electrode arrays subdurally implanted on the surface of the human cerebral cortex. While monkey LFPs showed a typical sequence of positive and negative potential peaks, an initial negative peak was the most salient feature of human EFPs. Individual contralateral LFPs from the monkey motor cortex carried approximately twice as much decoded information (DI) about arm movement direction (median 0.27 bit) as did individual EFPs from the contralateral hand/arm area of primary motor cortex in humans (median 0.12 bit). This relation was similar to the relation between median peak signal-to-noise ratios for directional modulation of movement related potentials (MRPs) of both types of signals. We discuss possible reasons for the observed differences, amongst them epi- vs. intracortical recording and the different electrode dimensions used to measure EFPs and LFPs.     

Meta level concept versus classic reflex concept for the control of posture and movement

Postural reflexes are replaced soon after birth by automatic reactions that allow for volition and cognition. It is still an enigma how this change in postural control is achieved. We suggest that the change involves the formation of a sensory processing level (meta level) that becomes interleaved in between the tight sensor-actuator coupling of the classic reflexes. We assume that the brain applies at this level intersensory interactions to reconstruct the physical stimuli which are causing the physiological stimuli and sensory signals. The thus derived estimates of the physical stimuli are then used as feedback signals in the posture control system. We present this concept on the background of the classic reflex concept and earlier attempts in the literature to overcome it. The earlier attempts were often motivated by the question how the brain prevents voluntary movements from being hampered by reflexive stabilisation of posture (so-called posture-movement problem). We compare our new concept with the classic reflex concept in a theoretical approach, by implementing both concepts into simple postural control models. In simulations of the two models we superimpose external perturbations (the physical stimuli) and a voluntary body lean movement. We show that it is possible to achieve successful stimulus compensation and unperturbed lean movement with both, the model derived from the new concept and the one of the classic reflex concept. With both approaches, the posture-movement problem does not arise. Based on preliminary considerations that include experimental findings from the literature, however, we conclude that the new concept provides more explanatory power than the classic reflex concept.     

Real-Time Subject-Independent Pattern Classification of Overt and Covert Movements from fNIRS Signals

Recently, studies have reported the use of Near Infrared Spectroscopy (NIRS) for developing Brain–Computer Interface (BCI) by applying online pattern classification of brain states from subject-specific fNIRS signals. The purpose of the present study was to develop and test a real-time method for subject-specific and subject-independent classification of multi-channel fNIRS signals using support-vector machines (SVM), so as to determine its feasibility as an online neurofeedback system. Towards this goal, we used left versus right hand movement execution and movement imagery as study paradigms in a series of experiments. In the first two experiments, activations in the motor cortex during movement execution and movement imagery were used to develop subject-dependent models that obtained high classification accuracies thereby indicating the robustness of our classification method. In the third experiment, a generalized classifier-model was developed from the first two experimental data, which was then applied for subject-independent neurofeedback training. Application of this method in new participants showed mean classification accuracy of 63% for movement imagery tasks and 80% for movement execution tasks. These results, and their corresponding offline analysis reported in this study demonstrate that SVM based real-time subject-independent classification of fNIRS signals is feasible. This method has important applications in the field of hemodynamic BCIs, and neuro-rehabilitation where patients can be trained to learn spatio-temporal patterns of healthy brain activity.     

The functions of the proprioceptors of the eye muscles

This article sets out to present a fairly comprehensive review of our knowledge about the functions of the receptors that have been found in the extraocular muscles--the six muscles that move each eye of vertebrates in its orbit--of all the animals in which they have been sought, including Man. Since their discovery at the beginning of the 20th century these receptors have, at various times, been credited with important roles in the control of eye movement and the construction of extrapersonal space and have also been denied any function whatsoever. Experiments intended to study the actions of eye muscle receptors and, even more so, opinions (and indeed polemic) derived from these observations have been influenced by the changing fashions and beliefs about the more general question of how limb position and movement is detected by the brain and which signals contribute to those aspects of this that are perceived (kinaesthesis). But the conclusions drawn from studies on the eye have also influenced beliefs about the mechanisms of kinaesthesis and, arguably, this influence has been even larger than that in the converse direction. Experimental evidence accumulated over rather more than a century is set out and discussed. It supports the view that, at the beginning of the 21st century, there are excellent grounds for believing that the receptors in the extraocular muscles are indeed proprioceptors, that is to say that the signals that they send into the brain are used to provide information about the position and movement of the eye in the orbit. It seems that this information is important in the control of eye movements of at least some types, and in the determination by the brain of the direction of gaze and the relationship of the organism to its environment. In addition, signals from these receptors in the eye muscles are seen to be necessary for the development of normal mechanisms of visual analysis in the mammalian visual cortex and for both the development and maintenance of normal visuomotor behaviour. Man is among those vertebrates to whose brains eye muscle proprioceptive signals provide information apparently used in normal sensorimotor functions; these include various aspects of perception, and of the control of eye movement. It is possible that abnormalities of the eye muscle proprioceptors and their signals may play a part in the genesis of some types of human squint (strabismus); conversely studies of patients with squint in the course of their surgical or pharmacological treatment have yielded much interesting evidence about the central actions of the proprioceptive signals from the extraocular muscles. The results of experiments on the eye have played a large part in the historical controversy, now in at least its third century, about the origin of signals that inform the brain about movement of parts of the body. Some of these results, and more of the interpretations of them, now need to be critically re-examined. The re-examination in the light of recent experiments that is presented here does not support many of the conclusions confidently drawn in the past and leads to both new insights and fresh questions about the roles of information from motor signals flowing out of the brain and that from signals from the peripheral receptors flowing into it. There remain many lacunae in our knowledge and filling some of these will, it is contended, be essential to advance our understanding further. It is argued that such understanding of eye muscle proprioception is a necessary part of the understanding of the physiology and pathophysiology of eye movement control and that it is also essential to an account of how organisms, including Man, build and maintain knowledge of their relationship to the external visual world. The eye would seem to provide a uniquely favourable system in which to study the way in which information derived within the brain about motor actions may interact with signals flowing in from peripheral receptors. The review is constructed in relatively independent sections that deal with particular topics. It ends with a fairly brief piece in which the author sets out some personal views about what has been achieved recently and what most immediately needs to be done. It also suggests some lines of study that appear to the author to be important for the future.     

Corollary discharge circuits in the primate brain

Movements are necessary to engage the world, but every movement results in sensorimotor ambiguity. Self-movements cause changes to sensory inflow as well as changes in the positions of objects relative to motor effectors (eyes and limbs). Hence the brain needs to monitor self-movements, and one way this is accomplished is by routing copies of movement commands to appropriate structures. These signals, known as corollary discharge (CD), enable compensation for sensory consequences of movement and preemptive updating of spatial representations. Such operations occur with a speed and accuracy that implies a reliance on prediction. Here we review recent CD studies and find that they arrive at a shared conclusion: CD contributes to prediction for the sake of sensorimotor harmony.     

Reconstructing visual experiences from brain activity evoked by natural movies

Quantitative modeling of human brain activity can provide crucial insights about cortical representations [1, 2] and can form the basis for brain decoding devices [3-5]. Recent functional magnetic resonance imaging (fMRI) studies have modeled brain activity elicited by static visual patterns and have reconstructed these patterns from brain activity [6-8]. However, blood oxygen level-dependent (BOLD) signals measured via fMRI are very slow [9], so it has been difficult to model brain activity elicited by dynamic stimuli such as natural movies. Here we present a new motion-energy [10, 11] encoding model that largely overcomes this limitation. The model describes fast visual information and slow hemodynamics by separate components. We recorded BOLD signals in occipitotemporal visual cortex of human subjects who watched natural movies and fit the model separately to individual voxels. Visualization of the fit models reveals how early visual areas represent the information in movies. To demonstrate the power of our approach, we also constructed a Bayesian decoder [8] by combining estimated encoding models with a sampled natural movie prior. The decoder provides remarkable reconstructions of the viewed movies. These results demonstrate that dynamic brain activity measured under naturalistic conditions can be decoded using current fMRI technology.     

Assessing Movement Factors in Upper Limb Kinematics Decoding from EEG Signals

The past decades have seen the rapid development of upper limb kinematics decoding techniques by performing intracortical recordings of brain signals. However, the use of non-invasive approaches to perform similar decoding procedures is still in its early stages. Recent studies show that there is a correlation between electroencephalographic (EEG) signals and hand-reaching kinematic parameters. From these studies, it could be concluded that the accuracy of upper limb kinematics decoding depends, at least partially, on the characteristics of the performed movement. In this paper, we have studied upper limb movements with different speeds and trajectories in a controlled environment to analyze the influence of movement variability in the decoding performance. To that end, low frequency components of the EEG signals have been decoded with linear models to obtain the position of the volunteer’s hand during performed trajectories grasping the end effector of a planar manipulandum. The results confirm that it is possible to obtain kinematic information from low frequency EEG signals and show that decoding performance is significantly influenced by movement variability and tracking accuracy as continuous and slower movements improve the accuracy of the decoder. This is a key factor that should be taken into account in future experimental designs.     

神经工程与脑-机接口

神经工程是近年来在生物医学工程领域备受关注的学科发展新方向。它运用神经科学和工程学的方法来分析神经功能并为神经功能缺失与紊乱的修复提供新的解决问题的方案;而脑-机接口则是当前神经工程领域中最活跃的研究方向之一。脑-机接口是在脑与计算机或其他外部设备之间建立的直接的通信和交流通道。在脑-机接口系统中,具有特定模式的脑信号携带着受试者希望表达的意愿,计算机将接收到的脑信号转换成相应的控制命令,于是那些有运动障碍的残疾人就可以利用脑-机接口系统来实现与外界的交流与对外部设备的控制。在基于脑电信号的脑-机接口系统中,受试者产生的脑信号大致可以分为内源性（endogenous）和外源性（exogenous）两类。其中外源性的成分主要取决于外部物理刺激（视觉、听觉或触觉）的参数而与认知行为无关;而内源性成分则主要由认知行为产生而与外部的物理刺激无关。在许多情况下,脑－机接口中的瞬态诱发电位通常都同时包含着内源性和外源性两种成分。寻找新的脑－机接口模式使之能显著提升记录脑电信号中的内源性与外源性成分在脑－机接口研究中具有重要意义。本文中将介绍一种基于运动起始时刻（motion—onset）的新的脑-机接口实验范式。本文的最后还探讨了脑－机接口未来发展的趋势与展望。     

Perivascular nerves and the regulation of cerebrovascular tone.

Brain perfusion is tightly coupled to neuronal activity, is commonly used to monitor normal or pathological brain function, and is a direct reflection of the interactions that occur between neuronal signals and blood vessels. Cerebral blood vessels at the surface and within the brain are surrounded by nerve fibers that originate, respectively, from peripheral nerve ganglia and intrinsic brain neurons. Although of different origin and targeting distinct vascular beds, these "perivascular nerves" fulfill similar roles related to cerebrovascular functions, a major one being to regulate their tone and, therein, brain perfusion. This utmost function, which underlies the signals used in functional neuroimaging techniques and which can be jeopardized in pathologies such as Alzheimer's disease, stroke, and migraine headache, is thus regulated at several levels. Recently, new insights into our understanding of how neural input regulate cerebrovascular tone resulted in the rediscovery of the functional "neurovascular unit." These remarkable advances suggest that neuron-driven changes in vascular tone result from interactions that involve all components of the neurovascular unit, transducing neuronal signals into vasomotor responses not only through direct interaction between neurons and vessels but also indirectly via the perivascular astrocytes. Neurovascular coupling is thus determined by chemical signals released from activated perivascular nerves and astrocytes that alter vascular tone to locally adjust perfusion to the spatial and temporal changes in brain activity.     

A Supplementary System for a Brain-Machine Interface Based on Jaw Artifacts for the Bidimensional Control of a Robotic Arm

Non-invasive Brain-Machine Interfaces (BMIs) are being used more and more these days to design systems focused on helping people with motor disabilities. Spontaneous BMIs translate user''s brain signals into commands to control devices. On these systems, by and large, 2 different mental tasks can be detected with enough accuracy. However, a large training time is required and the system needs to be adjusted on each session. This paper presents a supplementary system that employs BMI sensors, allowing the use of 2 systems (the BMI system and the supplementary system) with the same data acquisition device. This supplementary system is designed to control a robotic arm in two dimensions using electromyographical (EMG) signals extracted from the electroencephalographical (EEG) recordings. These signals are voluntarily produced by users clenching their jaws. EEG signals (with EMG contributions) were registered and analyzed to obtain the electrodes and the range of frequencies which provide the best classification results for 5 different clenching tasks. A training stage, based on the 2-dimensional control of a cursor, was designed and used by the volunteers to get used to this control. Afterwards, the control was extrapolated to a robotic arm in a 2-dimensional workspace. Although the training performed by volunteers requires 70 minutes, the final results suggest that in a shorter period of time (45 min), users should be able to control the robotic arm in 2 dimensions with their jaws. The designed system is compared with a similar 2-dimensional system based on spontaneous BMIs, and our system shows faster and more accurate performance. This is due to the nature of the control signals. Brain potentials are much more difficult to control than the electromyographical signals produced by jaw clenches. Additionally, the presented system also shows an improvement in the results compared with an electrooculographic system in a similar environment.