A symbiotic brain-machine interface through value-based decision making

Background

In the development of Brain Machine Interfaces (BMIs), there is a great need to enable users to interact with changing environments during the activities of daily life. It is expected that the number and scope of the learning tasks encountered during interaction with the environment as well as the pattern of brain activity will vary over time. These conditions, in addition to neural reorganization, pose a challenge to decoding neural commands for BMIs. We have developed a new BMI framework in which a computational agent symbiotically decoded users'' intended actions by utilizing both motor commands and goal information directly from the brain through a continuous Perception-Action-Reward Cycle (PARC).

Methodology

The control architecture designed was based on Actor-Critic learning, which is a PARC-based reinforcement learning method. Our neurophysiology studies in rat models suggested that Nucleus Accumbens (NAcc) contained a rich representation of goal information in terms of predicting the probability of earning reward and it could be translated into an evaluative feedback for adaptation of the decoder with high precision. Simulated neural control experiments showed that the system was able to maintain high performance in decoding neural motor commands during novel tasks or in the presence of reorganization in the neural input. We then implanted a dual micro-wire array in the primary motor cortex (M1) and the NAcc of rat brain and implemented a full closed-loop system in which robot actions were decoded from the single unit activity in M1 based on an evaluative feedback that was estimated from NAcc.

Conclusions

Our results suggest that adapting the BMI decoder with an evaluative feedback that is directly extracted from the brain is a possible solution to the problem of operating BMIs in changing environments with dynamic neural signals. During closed-loop control, the agent was able to solve a reaching task by capturing the action and reward interdependency in the brain.     

Bridging the brain to the world: a perspective on neural interface systems

Neural interface (NI) systems hold the potential to return lost functions to persons with paralysis. Impressive progress has been made, including evaluation of neural control signals, sensor testing in humans, signal decoding advances, and proof-of-concept validation. Most importantly, the field has demonstrated that persons with paralysis can use prototype systems for spelling, "point and click," and robot control. Human and animal NI research is advancing knowledge about neural information processing and plasticity in healthy, diseased, and injured nervous systems. This emerging field promises a range of neurotechnologies able to return communication, independence, and control to people with movement limitations.     

Encoder-Decoder Optimization for Brain-Computer Interfaces

Neuroprosthetic brain-computer interfaces are systems that decode neural activity into useful control signals for effectors, such as a cursor on a computer screen. It has long been recognized that both the user and decoding system can adapt to increase the accuracy of the end effector. Co-adaptation is the process whereby a user learns to control the system in conjunction with the decoder adapting to learn the user''s neural patterns. We provide a mathematical framework for co-adaptation and relate co-adaptation to the joint optimization of the user''s control scheme ("encoding model") and the decoding algorithm''s parameters. When the assumptions of that framework are respected, co-adaptation cannot yield better performance than that obtainable by an optimal initial choice of fixed decoder, coupled with optimal user learning. For a specific case, we provide numerical methods to obtain such an optimized decoder. We demonstrate our approach in a model brain-computer interface system using an online prosthesis simulator, a simple human-in-the-loop pyschophysics setup which provides a non-invasive simulation of the BCI setting. These experiments support two claims: that users can learn encoders matched to fixed, optimal decoders and that, once learned, our approach yields expected performance advantages.     

Coarse Electrocorticographic Decoding of Ipsilateral Reach in Patients with Brain Lesions

In patients with unilateral upper limb paralysis from strokes and other brain lesions, strategies for functional recovery may eventually include brain-machine interfaces (BMIs) using control signals from residual sensorimotor systems in the damaged hemisphere. When voluntary movements of the contralateral limb are not possible due to brain pathology, initial training of such a BMI may require use of the unaffected ipsilateral limb. We conducted an offline investigation of the feasibility of decoding ipsilateral upper limb movements from electrocorticographic (ECoG) recordings in three patients with different lesions of sensorimotor systems associated with upper limb control. We found that the first principal component (PC) of unconstrained, naturalistic reaching movements of the upper limb could be decoded from ipsilateral ECoG using a linear model. ECoG signal features yielding the best decoding accuracy were different across subjects. Performance saturated with very few input features. Decoding performances of 0.77, 0.73, and 0.66 (median Pearson''s r between the predicted and actual first PC of movement using nine signal features) were achieved in the three subjects. The performance achieved here with small numbers of electrodes and computationally simple decoding algorithms suggests that it may be possible to control a BMI using ECoG recorded from damaged sensorimotor brain systems.     

Signal acquisition and analysis for cortical control of neuroprosthetics

Work in cortically controlled neuroprosthetic systems has concentrated on decoding natural behaviors from neural activity, with the idea that if the behavior could be fully decoded it could be duplicated using an artificial system. Initial estimates from this approach suggested that a high-fidelity signal comprised of many hundreds of neurons would be required to control a neuroprosthetic system successfully. However, recent studies are showing hints that these systems can be controlled effectively using only a few tens of neurons. Attempting to decode the pre-existing relationship between neural activity and natural behavior is not nearly as important as choosing a decoding scheme that can be more readily deployed and trained to generate the desired actions of the artificial system. These artificial systems need not resemble or behave similarly to any natural biological system. Effective matching of discrete and continuous neural command signals to appropriately configured device functions will enable effective control of both natural and abstract artificial systems using compatible thought processes.     

An adaptive gaze stabilization controller inspired by the vestibulo-ocular reflex

The vestibulo-ocular reflex stabilizes vision in many vertebrates. It integrates inertial and visual information to drive the eyes in the opposite direction to head movement and thereby stabilizes the image on the retina. Its adaptive nature guarantees stable vision even when the biological system undergoes dynamic changes (due to disease, growth or fatigue etc), a characteristic especially desirable in autonomous robotic systems. Based on novel, biologically plausible neurological models, we have developed a robotic testbed to qualitatively evaluate the performance of these algorithms. We show how the adaptive controller can adapt to a time varying plant and elaborate how this biologically inspired control architecture can be employed in general engineering applications where sensory feedback is very noisy and/or delayed.     

Neuron selection by relative importance for neural decoding of dexterous finger prosthesis control application

Future generations of upper limb prosthesis will have dexterous hand with individual fingers and will be controlled directly by neural signals. Neurons from the primary motor (M1) cortex code for finger movements and provide the source for neural control of dexterous prosthesis. Each neuron's activation can be quantified by the change in firing rate before and after finger movement, and the quantified value is then represented by the neural activity over each trial for the intended movement. Since this neural activity varies with the intended movement, we define the relative importance of each neuron independent of specific intended movements. The relative importance of each neuron is determined by the inter-movement variance of the neural activities for respective intended movements. Neurons are ranked by the relative importance and then a subpopulation of rank-ordered neurons is selected for the neural decoding. The use of the proposed neuron selection method in individual finger movements improved decoding accuracy by 21.5% in the case of decoding with only 5 neurons and by 9.2% in the case of decoding with only 10 neurons. With only 15 highly ranked neurons, a decoding accuracy of 99.5% was achieved. The performance improvement is still maintained when combined movements of two fingers were included though the decoding accuracy fell to 95.7%. Since the proposed neuron selection method can achieve the targeting accuracy of decoding algorithms with less number of input neurons, it can be significant for developing brain–machine interfaces for direct neural control of hand prostheses.     

A real-time spiking cerebellum model for learning robot control

We describe a neural network model of the cerebellum based on integrate-and-fire spiking neurons with conductance-based synapses. The neuron characteristics are derived from our earlier detailed models of the different cerebellar neurons. We tested the cerebellum model in a real-time control application with a robotic platform. Delays were introduced in the different sensorimotor pathways according to the biological system. The main plasticity in the cerebellar model is a spike-timing dependent plasticity (STDP) at the parallel fiber to Purkinje cell connections. This STDP is driven by the inferior olive (IO) activity, which encodes an error signal using a novel probabilistic low frequency model. We demonstrate the cerebellar model in a robot control system using a target-reaching task. We test whether the system learns to reach different target positions in a non-destructive way, therefore abstracting a general dynamics model. To test the system's ability to self-adapt to different dynamical situations, we present results obtained after changing the dynamics of the robotic platform significantly (its friction and load). The experimental results show that the cerebellar-based system is able to adapt dynamically to different contexts.     

Towards On-line Adaptation of Neuro-prostheses with Neuronal Evaluation Signals

Many experiments have successfully demonstrated that prosthetic devices for restoring lost body functions can in principle be controlled by brain signals. However, stable long-term application of these devices, required for paralyzed patients, may suffer substantially from on-going signal changes for example adapting neural activities or movements of the electrodes recording brain activity. These changes currently require tedious re-learning procedures which are conducted and supervised under laboratory conditions, hampering the everyday use of such devices. As an efficient alternative to current methods we here propose an on-line adaptation scheme that exploits a hypothetical secondary signal source from brain regions reflecting the user’s affective evaluation of the current neuro- prosthetic’s performance. For demonstrating the feasibility of our idea, we simulate a typical prosthetic setup controlling a virtual robotic arm. Hereby we use the additional, hypothetical evaluation signal to adapt the decoding of the intended arm movement which is subjected to large non-stationarities. Even with weak signals and high noise levels typically encountered in recording brain activities, our simulations show that prosthetic devices can be adapted successfully during everyday usage, requiring no special training procedures. Furthermore, the adaptation is shown to be stable against large changes in neural encoding and/or in the recording itself.     

Virtual Reality Based Support System for Layout Planning and Programming of an Industrial Robotic Work Cell

Traditional robotic work cell design and programming are considered inefficient and outdated in current industrial and market demands. In this research, virtual reality (VR) technology is used to improve human-robot interface, whereby complicated commands or programming knowledge is not required. The proposed solution, known as VR-based Programming of a Robotic Work Cell (VR-Rocell), consists of two sub-programmes, which are VR-Robotic Work Cell Layout (VR-RoWL) and VR-based Robot Teaching System (VR-RoT). VR-RoWL is developed to assign the layout design for an industrial robotic work cell, whereby VR-RoT is developed to overcome safety issues and lack of trained personnel in robot programming. Simple and user-friendly interfaces are designed for inexperienced users to generate robot commands without damaging the robot or interrupting the production line. The user is able to attempt numerous times to attain an optimum solution. A case study is conducted in the Robotics Laboratory to assemble an electronics casing and it is found that the output models are compatible with commercial software without loss of information. Furthermore, the generated KUKA commands are workable when loaded into a commercial simulator. The operation of the actual robotic work cell shows that the errors may be due to the dynamics of the KUKA robot rather than the accuracy of the generated programme. Therefore, it is concluded that the virtual reality based solution approach can be implemented in an industrial robotic work cell.     

Visually guided gait modifications for stepping over an obstacle: a bio-inspired approach

There is an increasing interest in conceiving robotic systems that are able to move and act in an unstructured and not predefined environment, for which autonomy and adaptability are crucial features. In nature, animals are autonomous biological systems, which often serve as bio-inspiration models, not only for their physical and mechanical properties, but also their control structures that enable adaptability and autonomy—for which learning is (at least) partially responsible. This work proposes a system which seeks to enable a quadruped robot to online learn to detect and to avoid stumbling on an obstacle in its path. The detection relies in a forward internal model that estimates the robot’s perceptive information by exploring the locomotion repetitive nature. The system adapts the locomotion in order to place the robot optimally before attempting to step over the obstacle, avoiding any stumbling. Locomotion adaptation is achieved by changing control parameters of a central pattern generator (CPG)-based locomotion controller. The mechanism learns the necessary alterations to the stride length in order to adapt the locomotion by changing the required CPG parameter. Both learning tasks occur online and together define a sensorimotor map, which enables the robot to learn to step over the obstacle in its path. Simulation results show the feasibility of the proposed approach.     

A robotic device for understanding neuromechanical interactions during standing balance control

Postural stability in standing balance results from the mechanics of body dynamics as well as active neural feedback control processes. Even when an animal or human has multiple legs on the ground, active neural regulation of balance is required. When the postural configuration, or stance, changes, such as when the feet are placed further apart, the mechanical stability of the organism changes, but the degree to which this alters the demands on neural feedback control for postural stability is unknown. We developed a robotic system that mimics the neuromechanical postural control system of a cat in response to lateral perturbations. This simple robotic system allows us to study the interactions between various parameters that contribute to postural stability and cannot be independently varied in biological systems. The robot is a 'planar', two-legged device that maintains compliant balance control in a variety of stance widths when subject to perturbations of the support surface, and in this sense reveals principles of lateral balance control that are also applicable to bipeds. Here we demonstrate that independent variations in either stance width or delayed neural feedback gains can have profound and often surprisingly detrimental effects on the postural stability of the system. Moreover, we show through experimentation and analysis that changing stance width alters fundamental mechanical relationships important in standing balance control and requires a coordinated adjustment of delayed feedback control to maintain postural stability.     

Decoding Hindlimb Movement for a Brain Machine Interface after a Complete Spinal Transection

Stereotypical locomotor movements can be made without input from the brain after a complete spinal transection. However, the restoration of functional gait requires descending modulation of spinal circuits to independently control the movement of each limb. To evaluate whether a brain-machine interface (BMI) could be used to regain conscious control over the hindlimb, rats were trained to press a pedal and the encoding of hindlimb movement was assessed using a BMI paradigm. Off-line, information encoded by neurons in the hindlimb sensorimotor cortex was assessed. Next neural population functions, or weighted representations of the neuronal activity, were used to replace the hindlimb movement as a trigger for reward in real-time (on-line decoding) in three conditions: while the animal could still press the pedal, after the pedal was removed and after a complete spinal transection. A novel representation of the motor program was learned when the animals used neural control to achieve water reward (e.g. more information was conveyed faster). After complete spinal transection, the ability of these neurons to convey information was reduced by more than 40%. However, this BMI representation was relearned over time despite a persistent reduction in the neuronal firing rate during the task. Therefore, neural control is a general feature of the motor cortex, not restricted to forelimb movements, and can be regained after spinal injury.     

In Good Company? Perception of Movement Synchrony of a Non-Anthropomorphic Robot

Recent technological developments like cheap sensors and the decreasing costs of computational power have brought the possibility of robotic home companions within reach. In order to be accepted it is vital for these robots to be able to participate meaningfully in social interactions with their users and to make them feel comfortable during these interactions. In this study we investigated how people respond to a situation where a companion robot is watching its user. Specifically, we tested the effect of robotic behaviours that are synchronised with the actions of a human. We evaluated the effects of these behaviours on the robot’s likeability and perceived intelligence using an online video survey. The robot used was Care-O-bot3, a non-anthropomorphic robot with a limited range of expressive motions. We found that even minimal, positively synchronised movements during an object-oriented task were interpreted by participants as engagement and created a positive disposition towards the robot. However, even negatively synchronised movements of the robot led to more positive perceptions of the robot, as compared to a robot that does not move at all. The results emphasise a) the powerful role that robot movements in general can have on participants’ perception of the robot, and b) that synchronisation of body movements can be a powerful means to enhance the positive attitude towards a non-anthropomorphic robot.     

Adaptive cerebellar spiking model embedded in the control loop: context switching and robustness against noise

This work evaluates the capability of a spiking cerebellar model embedded in different loop architectures (recurrent, forward, and forward&recurrent) to control a robotic arm (three degrees of freedom) using a biologically-inspired approach. The implemented spiking network relies on synaptic plasticity (long-term potentiation and long-term depression) to adapt and cope with perturbations in the manipulation scenario: changes in dynamics and kinematics of the simulated robot. Furthermore, the effect of several degrees of noise in the cerebellar input pathway (mossy fibers) was assessed depending on the employed control architecture. The implemented cerebellar model managed to adapt in the three control architectures to different dynamics and kinematics providing corrective actions for more accurate movements. According to the obtained results, coupling both control architectures (forward&recurrent) provides benefits of the two of them and leads to a higher robustness against noise.     

Efficient Universal Computing Architectures for Decoding Neural Activity

The ability to decode neural activity into meaningful control signals for prosthetic devices is critical to the development of clinically useful brain– machine interfaces (BMIs). Such systems require input from tens to hundreds of brain-implanted recording electrodes in order to deliver robust and accurate performance; in serving that primary function they should also minimize power dissipation in order to avoid damaging neural tissue; and they should transmit data wirelessly in order to minimize the risk of infection associated with chronic, transcutaneous implants. Electronic architectures for brain– machine interfaces must therefore minimize size and power consumption, while maximizing the ability to compress data to be transmitted over limited-bandwidth wireless channels. Here we present a system of extremely low computational complexity, designed for real-time decoding of neural signals, and suited for highly scalable implantable systems. Our programmable architecture is an explicit implementation of a universal computing machine emulating the dynamics of a network of integrate-and-fire neurons; it requires no arithmetic operations except for counting, and decodes neural signals using only computationally inexpensive logic operations. The simplicity of this architecture does not compromise its ability to compress raw neural data by factors greater than . We describe a set of decoding algorithms based on this computational architecture, one designed to operate within an implanted system, minimizing its power consumption and data transmission bandwidth; and a complementary set of algorithms for learning, programming the decoder, and postprocessing the decoded output, designed to operate in an external, nonimplanted unit. The implementation of the implantable portion is estimated to require fewer than 5000 operations per second. A proof-of-concept, 32-channel field-programmable gate array (FPGA) implementation of this portion is consequently energy efficient. We validate the performance of our overall system by decoding electrophysiologic data from a behaving rodent.     

A Pilot Study on the Feasibility of Robot-Aided Leg Motor Training to Facilitate Active Participation

Robot-aided gait therapy offers a promising approach towards improving gait function in individuals with neurological disorders such as stroke or spinal cord injury. However, incorporation of appropriate control strategies is essential for actively engaging the patient in the therapeutic process. Although several control algorithms (such as assist-as-needed and error augmentation) have been proposed to improve active patient participation, we hypothesize that the therapeutic benefits of these control algorithms can be greatly enhanced if combined with a motor learning task to facilitate neural reorganization and motor recovery. Here, we describe an active robotic training approach (patient-cooperative robotic gait training combined with a motor learning task) using the Lokomat and pilot-tested whether this approach can enhance active patient participation during training. Six neurologically intact adults and three chronic stroke survivors participated in this pilot feasibility study. Participants walked in a Lokomat while simultaneously performing a foot target-tracking task that necessitated greater hip and knee flexion during the swing phase of the gait. We computed the changes in tracking error as a measure of motor performance and changes in muscle activation as a measure of active subject participation. Repeated practice of the motor-learning task resulted in significant reductions in target-tracking error in all subjects. Muscle activation was also significantly higher during active robotic training compared to simply walking in the robot. The data from stroke participants also showed a trend similar to neurologically intact participants. These findings provide a proof-of-concept demonstration that combining robotic gait training with a motor learning task enhances active participation.     

The Treeterbi and Parallel Treeterbi algorithms: efficient, optimal decoding for ordinary, generalized and pair HMMs

MOTIVATION: Hidden Markov models (HMMs) and generalized HMMs been successfully applied to many problems, but the standard Viterbi algorithm for computing the most probable interpretation of an input sequence (known as decoding) requires memory proportional to the length of the sequence, which can be prohibitive. Existing approaches to reducing memory usage either sacrifice optimality or trade increased running time for reduced memory. RESULTS: We developed two novel decoding algorithms, Treeterbi and Parallel Treeterbi, and implemented them in the TWINSCAN/N-SCAN gene-prediction system. The worst case asymptotic space and time are the same as for standard Viterbi, but in practice, Treeterbi optimally decodes arbitrarily long sequences with generalized HMMs in bounded memory without increasing running time. Parallel Treeterbi uses the same ideas to split optimal decoding across processors, dividing latency to completion by approximately the number of available processors with constant average overhead per processor. Using these algorithms, we were able to optimally decode all human chromosomes with N-SCAN, which increased its accuracy relative to heuristic solutions. We also implemented Treeterbi for Pairagon, our pair HMM based cDNA-to-genome aligner. AVAILABILITY: The TWINSCAN/N-SCAN/PAIRAGON open source software package is available from http://genes.cse.wustl.edu.     

Putting it together: the computational complexity of designing robot controllers and environments for distributed construction

Creating target structures through the coordinated efforts of teams of autonomous robots (possibly aided by specific features in their environments) is a very important problem in distributed robotics. Many specific instances of distributed robotic construction teams have been developed manually. An important issue is whether automated controller design algorithms can both quickly produce robot controllers and guarantee that teams using these controllers will build arbitrary requested target structures correctly; this task may also involve specifying features in the environment that can aid the construction process. In this paper, we give the first computational and parameterized complexity analyses of several problems associated with the design of robot controllers and environments for creating target structures. These problems use a simple finite-state robot controller model that moves in a non-continuous deterministic manner in a grid-based environment. Our goal is to establish whether algorithms exist that are both fast and correct for all inputs and if not, under which restrictions such algorithms are possible. We prove that none of these problems are efficiently solvable in general and remain so under a number of plausible restrictions on controllers, environments, and target structures. We also give the first restrictions relative to which these problems are efficiently solvable and discuss what theoretical solvability and unsolvability results derived relative to the problems examined here mean for real-world construction using robot teams.     

A Brain-Machine-Muscle Interface for Restoring Hindlimb Locomotion after Complete Spinal Transection in Rats

A brain-machine interface (BMI) is a neuroprosthetic device that can restore motor function of individuals with paralysis. Although the feasibility of BMI control of upper-limb neuroprostheses has been demonstrated, a BMI for the restoration of lower-limb motor functions has not yet been developed. The objective of this study was to determine if gait-related information can be captured from neural activity recorded from the primary motor cortex of rats, and if this neural information can be used to stimulate paralysed hindlimb muscles after complete spinal cord transection. Neural activity was recorded from the hindlimb area of the primary motor cortex of six female Sprague Dawley rats during treadmill locomotion before and after mid-thoracic transection. Before spinal transection there was a strong association between neural activity and the step cycle. This association decreased after spinal transection. However, the locomotive state (standing vs. walking) could still be successfully decoded from neural recordings made after spinal transection. A novel BMI device was developed that processed this neural information in real-time and used it to control electrical stimulation of paralysed hindlimb muscles. This system was able to elicit hindlimb muscle contractions that mimicked forelimb stepping. We propose this lower-limb BMI as a future neuroprosthesis for human paraplegics.