A biologically inspired neural net for trajectory formation and obstacle avoidance

In this paper we present a biologically inspired two-layered neural network for trajectory formation and obstacle avoidance. The two topographically ordered neural maps consist of analog neurons having continuous dynamics. The first layer, the sensory map, receives sensory information and builds up an activity pattern which contains the optimal solution (i.e. shortest path without collisions) for any given set of current position, target positions and obstacle positions. Targets and obstacles are allowed to move, in which case the activity pattern in the sensory map will change accordingly. The time evolution of the neural activity in the second layer, the motor map, results in a moving cluster of activity, which can be interpreted as a population vector. Through the feedforward connections between the two layers, input of the sensory map directs the movement of the cluster along the optimal path from the current position of the cluster to the target position. The smooth trajectory is the result of the intrinsic dynamics of the network only. No supervisor is required. The output of the motor map can be used for direct control of an autonomous system in a cluttered environment or for control of the actuators of a biological limb or robot manipulator. The system is able to reach a target even in the presence of an external perturbation. Computer simulations of a point robot and a multi-joint manipulator illustrate the theory.     

Neuroethological approaches to the study of motor development in chicks: achievements and challenges.

Chicks and chick embryos provide a useful model system for the study of issues related to the development of motor behaviors. EMG and kinematic analyses of leg movements have been used to provide new data on the organization of embryonic motility. These data suggest that the circuitry needed to produce a basic, coordinated motor pattern is available early in development. This circuitry then appears to be retained throughout life. Evidence from analysis of EMG patterns and leg deafferentation studies suggest that the output of this basic circuit can be modulated by sensory input to produce the motor patterns of later behaviors, such as hatching and walking. If the same circuitry is present throughout life, then mechanisms for initiation and termination of particular behaviors must be available to ensure that specific behaviors are turned on and off at appropriate times. For example, hatching can be turned on by a specific sensory signal: proprioceptive signals from the bent neck. In addition to reviewing current research on the development of chick motor behaviors, methodological considerations and suggestions for future research are presented.     

Neuroethological approaches to the study of motor development in chicks: Achievements and challenges

Chicks and chick embryos provide a useful model system for the study of related to the development of motor behaviors. EMG and kinematic analyses of leg movements have been used to provide new data on the organization of embryonic motility. These data suggest that the circuitry needed to produce a basic, coordinated motor pattern is available early in development. This circuitry then appears to be retained throughout life. Evidence from analysis of EMG patterns and leg deafferentation studies suggest that the output of this basic circuit can be modulated by sensory input to produce the motor patterns of later behaviors, such as hatching and walking. If the same circuitry is present throughout life, then mechanisms for initiation and termination of particular behaviors must be available to ensure that specific behaviors are turned on and off at appropriate times. For example, hatching can be turned on by a specific sensory signal: proprioceptive signals from the bent neck. In addition to reviewing current research on the development of chick motor behaviors, methodological considerations and suggestions for future research are presented. © 1992 John Wiley & Sons, Inc.     

Spinal motor control system incorporates an internal model of limb dynamics

The existence and utilization of an internal representation of the controlled object is one of the most important features of the functioning of neural motor control systems. This study demonstrates that this property already exists at the level of the spinal motor control system (SMCS), which is capable of generating motor patterns for reflex rhythmic movements, such as locomotion and scratching, without the aid of the peripheral afferent feedback, but substantially modifies the generated activity in response to peripheral afferent stimuli. The SMCS is presented as an optimal control system whose optimality requires that it incorporate an internal model (IM) of the controlled object's dynamics. A novel functional mechanism for the integration of peripheral sensory signals with the corresponding predictive output from the IM, the summation of information precision (SIP) is proposed. In contrast to other models in which the correction of the internal representation of the controlled object's state is based on the calculation of a mismatch between the internal and external information sources, the SIP mechanism merges the information from these sources in order to optimize the precision of the controlled object's state estimate. It is demonstrated, based on scratching in decerebrate cats as an example of the spinal control of goal-directed movements, that the results of computer modeling agree with the experimental observations related to the SMCS's reactions to phasic and tonic peripheral afferent stimuli. It is also shown that the functional requirements imposed by the mathematical model of the SMCS comply with the current knowledge about the related properties of spinal neuronal circuitry. The crucial role of the spinal presynaptic inhibition mechanism in the neuronal implementation of SIP is elucidated. Important differences between the IM and a state predictor employed for compensating for a neural reflex time delay are discussed. Received: 8 February 2000 / Accepted: 24 March 2000     

A model for control of breathing in mammals: coupling neural dynamics to peripheral gas exchange and transport

A new model for aspects of the control of respiration in mammals has been developed. The model integrates a reduced representation of the brainstem respiratory neural controller together with peripheral gas exchange and transport mechanisms. The neural controller consists of two components. One component represents the inspiratory oscillator in the pre-Bötzinger complex (pre-BötC) incorporating biophysical mechanisms for rhythm generation. The other component represents the ventral respiratory group (VRG), which is driven by the pre-BötC for generation of inspiratory (pre)motor output. The neural model was coupled to simplified models of the lungs incorporating oxygen and carbon dioxide transport. The simplified representation of the brainstem neural circuitry has regulation of both frequency and amplitude of respiration and is done in response to partial pressures of oxygen and carbon dioxide in the blood using proportional (P) and proportional plus integral (PI) controllers. We have studied the coupled system under open and closed loop control. We show that two breathing regimes can exist in the model. In one regime an increase in the inspiratory frequency is accompanied by an increase in amplitude. In the second regime an increase in frequency is accompanied by a decrease in amplitude. The dynamic response of the model to changes in the concentration of inspired O₂ or inspired CO₂ was compared qualitatively with experimental data reported in the physiological literature. We show that the dynamic response with a PI-controller fits the experimental data better but suggests that when high levels of CO₂ are inspired the respiratory system cannot reach steady state. Our model also predicts that there could be two possible mechanisms for apnea appearance when 100% O₂ is inspired following a period of 5% inspired O₂. This paper represents a novel attempt to link neural control and gas transport mechanisms, highlights important issues in amplitude and frequency control and sets the stage for more complete neurophysiological control models.     

Modeling inter-human movement coordination: synchronization governs joint task dynamics

Human interaction partners tend to synchronize their movements during repetitive actions such as walking. Research of inter-human coordination in purely rhythmic action tasks reveals that the observed patterns of interaction are dominated by synchronization effects. Initiated by our finding that human dyads synchronize their arm movements even in a goal-directed action task, we present a step-wise approach to a model of inter-human movement coordination. In an experiment, the hand trajectories of ten human dyads are recorded. Governed by a dynamical process of phase synchronization, the participants establish in-phase as well as anti-phase relations. The emerging relations are successfully reproduced by the attractor dynamics of coupled phase oscillators inspired by the Kuramoto model. Three different methods on transforming the motion trajectories into instantaneous phases are investigated and their influence on the model fit to the experimental data is evaluated. System identification technique allows us to estimate the model parameters, which are the coupling strength and the frequency detuning among the dyad. The stability properties of the identified model match the relations observed in the experimental data. In short, our model predicts the dynamics of inter-human movement coordination. It can directly be implemented to enrich human-robot interaction.     

Human control of a simple two-hand grasp

We investigated how people control fast, accurate movements of a load using a simple two-hand grasp. By providing a clear instruction to several subjects, we isolated a single control strategy. The kinematics produced by this control strategy are nearly indistinguishable from those produced during single-hand movements, but the torques are quite different: one hand accelerates not only itself, but also the load and the other hand, while the other hand brakes the hand-load-hand system. As a result, the hands squeeze the load with a large force during the movement.The dynamics of the hand-load-hand system are of the same form as the dynamics of a single-hand system. Apparently, by taking advantage of this dynamic similarity and of the spring-like properties of muscle, the human motor control system can control the two-hand grasp system simply by modifying the muscle activation patterns used to control single-hand movements.The task dynamics of two-hand grasp do not require that the load be squeezed during the movement, and squeezing the load wastes torque that could be used to move more quickly. However, the human motor control system may choose this squeezing strategy because it reliably brakes the hand-load-hand system despite inherent variability in the braking of individual hands.     

Functional organization of adult motor cortex is dependent upon continued protein synthesis

The functional organization of adult cerebral cortex is characterized by the presence of highly ordered sensory and motor maps. Despite their archetypical organization, the maps maintain the capacity to rapidly reorganize, suggesting that the neural circuitry underlying cortical representations is inherently plastic. Here we show that the circuitry supporting motor maps is dependent upon continued protein synthesis. Injections of two different protein synthesis inhibitors into adult rat forelimb motor cortex caused an immediate and enduring loss of movement representations. The disappearance of the motor map was accompanied by a significant reduction in synapse number, synapse size, and cortical field potentials and caused skilled forelimb movement impairments. Further, motor skill training led to a reappearance of movement representations. We propose that the circuitry of adult motor cortex is perpetually labile and requires continued protein synthesis in order to maintain its functional organization.     

Plasticity of functional connectivity in the adult spinal cord

This paper emphasizes several characteristics of the neural control of locomotion that provide opportunities for developing strategies to maximize the recovery of postural and locomotor functions after a spinal cord injury (SCI). The major points of this paper are: (i) the circuitry that controls standing and stepping is extremely malleable and reflects a continuously varying combination of neurons that are activated when executing stereotypical movements; (ii) the connectivity between neurons is more accurately perceived as a functional rather than as an anatomical phenomenon; (iii) the functional connectivity that controls standing and stepping reflects the physiological state of a given assembly of synapses, where the probability of these synaptic events is not deterministic; (iv) rather, this probability can be modulated by other factors such as pharmacological agents, epidural stimulation and/or motor training; (v) the variability observed in the kinematics of consecutive steps reflects a fundamental feature of the neural control system and (vi) machine-learning theories elucidate the need to accommodate variability in developing strategies designed to enhance motor performance by motor training using robotic devices after an SCI.     

Computational nature of human adaptive control during learning of reaching movements in force fields

Learning to make reaching movements in force fields was used as a paradigm to explore the system architecture of the biological adaptive controller. We compared the performance of a number of candidate control systems that acted on a model of the neuromuscular system of the human arm and asked how well the dynamics of the candidate system compared with the movement characteristics of 16 subjects. We found that control via a supra-spinal system that utilized an adaptive inverse model resulted in dynamics that were similar to that observed in our subjects, but lacked essential characteristics. These characteristics pointed to a different architecture where descending commands were influenced by an adaptive forward model. However, we found that control via a forward model alone also resulted in dynamics that did not match the behavior of the human arm. We considered a third control architecture where a forward model was used in conjunction with an inverse model and found that the resulting dynamics were remarkably similar to that observed in the experimental data. The essential property of this control architecture was that it predicted a complex pattern of near-discontinuities in hand trajectory in the novel force field. A nearly identical pattern was observed in our subjects, suggesting that generation of descending motor commands was likely through a control system architecture that included both adaptive forward and inverse models. We found that as subjects learned to make reaching movements, adaptation rates for the forward and inverse models could be independently estimated and the resulting changes in performance of subjects from movement to movement could be accurately accounted for. Results suggested that the adaptation of the forward model played a dominant role in the motor learning of subjects. After a period of consolidation, the rates of adaptation in the internal models were significantly larger than those observed before the memory had consolidated. This suggested that consolidation of motor memory coincided with freeing of certain computational resources for subsequent learning. Received: 01 January 1998 / Accepted in revised form: 26 January 1999     

Sensorimotor regulation of movements: Novel strategies for the recovery of mobility

A series of observations have provided important insight into properties of the spinal as well as supraspinal circuitries that control posture and movement. We have demonstrated that spinal rats can regain full weight-bearing standing and stepping over a range of speeds and directions with the aid of electrically enabling motor control (eEmc), pharmacological modulation (fEmc), and training [1, 2]. Also, we have reported that voluntary control movements of individual joints and limbs can be regained after complete paralysis in humans [3, 4]. However, the ability to generate significant levels of voluntary weight-bearing stepping with or without epidural spinal cord stimulation remains limited. Herein we introduce a novel method of painless transcutaneous electrical enabling motor control (pcEmc) and sensory enabling motor control (sEmc) strategy to neuromodulate the physiological state of the spinal cord. We have found that a combination of a novel non-invasive transcutaneous spinal cord stimulation and sensory-motor stimulation of leg mechanoreceptors can modulate the spinal locomotor circuitry to state that enables voluntary rhythmic locomotor movements.     

Cerebellar learning of accurate predictive control for fast-reaching movements

Long conduction delays in the nervous system prevent the accurate control of movements by feedback control alone. We present a new, biologically plausible cerebellar model to study how fast arm movements can be executed in spite of these delays. To provide a realistic test-bed of the cerebellar neural model, we embed the cerebellar network in a simulated biological motor system comprising a spinal cord model and a six-muscle two-dimensional arm model. We argue that if the trajectory errors are detected at the spinal cord level, memory traces in the cerebellum can solve the temporal mismatch problem between efferent motor commands and delayed error signals. Moreover, learning is made stable by the inclusion of the cerebello-nucleo-olivary loop in the model. It is shown that the cerebellar network implements a nonlinear predictive regulator by learning part of the inverse dynamics of the plant and spinal circuit. After learning, fast accurate reaching movements can be generated. Received: 8 February 1999 /Accepted in revised form: 7 August 1999     

An optimisation-based model for full-body upright reaching movements

An optimal simulation 3D model for full-body upright reaching movements was developed using graphic-based modelling tools (SimMechanics) to generate an inverse dynamics model of the skeleton and using parameterisation methods for a sensory motor controller. The adaptive weight coefficient of the cost function based on the final motor task error (i.e. distance between end-effector and target at the end of movement) was used to correct motor task error and physiological measurements (e.g. joint power, centre of mass displacement, etc.). The output of the simulation models using various cost functions were compared to experimental data from 15 healthy participants performing full-body upright reaching movements. The proposed method can reasonably predict full-body voluntary movements in terms of final posture, joint power, and movement of the centre of mass (COM) using simple algebraic calculations of inverse dynamics and forward kinematics instead of the complicated integrals of the forward dynamics. We found that the combination of several control strategies, i.e. minimising end-effector error, total joint power and body COM produced the best fit of the full-body reaching task.     

Emergence of a Stable Cortical Map for Neuroprosthetic Control

Cortical control of neuroprosthetic devices is known to require neuronal adaptations. It remains unclear whether a stable cortical representation for prosthetic function can be stored and recalled in a manner that mimics our natural recall of motor skills. Especially in light of the mixed evidence for a stationary neuron-behavior relationship in cortical motor areas, understanding this relationship during long-term neuroprosthetic control can elucidate principles of neural plasticity as well as improve prosthetic function. Here, we paired stable recordings from ensembles of primary motor cortex neurons in macaque monkeys with a constant decoder that transforms neural activity to prosthetic movements. Proficient control was closely linked to the emergence of a surprisingly stable pattern of ensemble activity, indicating that the motor cortex can consolidate a neural representation for prosthetic control in the presence of a constant decoder. The importance of such a cortical map was evident in that small perturbations to either the size of the neural ensemble or to the decoder could reversibly disrupt function. Moreover, once a cortical map became consolidated, a second map could be learned and stored. Thus, long-term use of a neuroprosthetic device is associated with the formation of a cortical map for prosthetic function that is stable across time, readily recalled, resistant to interference, and resembles a putative memory engram.     

Motor training induces experience-specific patterns of plasticity across motor cortex and spinal cord.

The motor cortex and spinal cord possess the remarkable ability to alter structure and function in response to differential motor training. Here we review the evidence that the corticospinal system is not only plastic but that the nature and locus of this plasticity is dictated by the specifics of the motor experience. Skill training induces synaptogenesis, synaptic potentiation, and reorganization of movement representations within motor cortex. Endurance training induces angiogenesis in motor cortex, but it does not alter motor map organization or synapse number. Strength training alters spinal motoneuron excitability and induces synaptogenesis within spinal cord, but it does not alter motor map organization. All three training experiences induce changes in spinal reflexes that are dependent on the specific behavioral demands of the task. These results demonstrate that the acquisition of skilled movement induces a reorganization of neural circuitry within motor cortex that supports the production and refinement of skilled movement sequences. We present data that suggest increases in strength may be mediated by an increased capacity for activation and/or recruitment of spinal motoneurons while the increased metabolic demands associated with endurance training induce cortical angiogenesis. Together these results show the robust pattern of anatomic and physiological plasticity that occurs within the corticospinal system in response to differential motor experience. The consequences of such distributed, experience-specific plasticity for the encoding of motor experience by the motor system are discussed.     

Distinct motor plans form and retrieve distinct motor memories for physically identical movements

We can adapt movements to a novel dynamic environment (e.g., tool use, microgravity, and perturbation) by acquiring an internal model of the dynamics. Although multiple environments can be learned simultaneously if each environment is experienced with different limb movement kinematics, it is controversial as to whether multiple internal models for a particular movement can be learned and flexibly retrieved according to behavioral contexts. Here, we address this issue by using a novel visuomotor task. While participants reached to each of two targets located at a clockwise or counter-clockwise position, a gradually increasing visual rotation was applied in the clockwise or counter-clockwise direction, respectively, to the on-screen cursor representing the unseen hand position. This procedure implicitly led participants to perform physically identical pointing movements irrespective of their intentions (i.e., movement plans) to move their hand toward two distinct visual targets. Surprisingly, if each identical movement was executed according to a distinct movement plan, participants could readily adapt these movements to two opposing force fields simultaneously. The results demonstrate that multiple motor memories can be learned and flexibly retrieved, even for physically identical movements, according to distinct motor plans in a visual space.     

A hierarchical neural-network model for control and learning of voluntary movement

In order to control voluntary movements, the central nervous system (CNS) must solve the following three computational problems at different levels: the determination of a desired trajectory in the visual coordinates, the transformation of its coordinates to the body coordinates and the generation of motor command. Based on physiological knowledge and previous models, we propose a hierarchical neural network model which accounts for the generation of motor command. In our model the association cortex provides the motor cortex with the desired trajectory in the body coordinates, where the motor command is then calculated by means of long-loop sensory feedback. Within the spinocerebellum — magnocellular red nucleus system, an internal neural model of the dynamics of the musculoskeletal system is acquired with practice, because of the heterosynaptic plasticity, while monitoring the motor command and the results of movement. Internal feedback control with this dynamical model updates the motor command by predicting a possible error of movement. Within the cerebrocerebellum — parvocellular red nucleus system, an internal neural model of the inverse-dynamics of the musculo-skeletal system is acquired while monitoring the desired trajectory and the motor command. The inverse-dynamics model substitutes for other brain regions in the complex computation of the motor command. The dynamics and the inverse-dynamics models are realized by a parallel distributed neural network, which comprises many sub-systems computing various nonlinear transformations of input signals and a neuron with heterosynaptic plasticity (that is, changes of synaptic weights are assumed proportional to a product of two kinds of synaptic inputs). Control and learning performance of the model was investigated by computer simulation, in which a robotic manipulator was used as a controlled system, with the following results: (1) Both the dynamics and the inverse-dynamics models were acquired during control of movements. (2) As motor learning proceeded, the inverse-dynamics model gradually took the place of external feedback as the main controller. Concomitantly, overall control performance became much better. (3) Once the neural network model learned to control some movement, it could control quite different and faster movements. (4) The neural netowrk model worked well even when only very limited information about the fundamental dynamical structure of the controlled system was available. Consequently, the model not only accounts for the learning and control capability of the CNS, but also provides a promising parallel-distributed control scheme for a large-scale complex object whose dynamics are only partially known.     

The origin of behavioral bursts in decision-making circuitry

From ants to humans, the timing of many animal behaviors comes in bursts of activity separated by long periods of inactivity. Recently, mathematical modeling has shown that simple algorithms of priority-driven behavioral choice can result in bursty behavior. To experimentally test this link between decision-making circuitry and bursty dynamics, we have turned to Drosophila melanogaster. We have found that the statistics of intervals between activity periods in endogenous activity-rest switches of wild-type Drosophila are very well described by the Weibull distribution, a common distribution of bursty dynamics in complex systems. The bursty dynamics of wild-type Drosophila walking activity are shown to be determined by this inter-event distribution alone and not by memory effects, thus resembling human dynamics. Further, using mutant flies that disrupt dopaminergic signaling or the mushroom body, circuitry implicated in decision-making, we show that the degree of behavioral burstiness can be modified. These results are thus consistent with the proposed link between decision-making circuitry and bursty dynamics, and highlight the importance of using simple experimental systems to test general theoretical models of behavior. The findings further suggest that analysis of bursts could prove useful for the study and evaluation of decision-making circuitry.