Energy efficient walking with central pattern generators: from passive dynamic walking to biologically inspired control

Like human walking, passive dynamic walking—i.e. walking down a slope with no actuation except gravity—is energy efficient by exploiting the natural dynamics. In the animal world, neural oscillators termed central pattern generators (CPGs) provide the basic rhythm for muscular activity in locomotion. We present a CPG model, which automatically tunes into the resonance frequency of the passive dynamics of a bipedal walker, i.e. the CPG model exhibits resonance tuning behavior. Each leg is coupled to its own CPG, controlling the hip moment of force. Resonance tuning above the endogenous frequency of the CPG—i.e. the CPG’s eigenfrequency—is achieved by feedback of both limb angles to their corresponding CPG, while integration of the limb angles provides resonance tuning at and below the endogenous frequency of the CPG. Feedback of the angular velocity of both limbs to their corresponding CPG compensates for the time delay in the loop coupling each limb to its CPG. The resonance tuning behavior of the CPG model allows the gait velocity to be controlled by a single parameter, while retaining the energy efficiency of passive dynamic walking.     

A comparison of resonance tuning with positive versus negative sensory feedback

We used a computational model of rhythmic movement to analyze how the connectivity of sensory feedback affects the tuning of a closed-loop neuromechanical system to the mechanical resonant frequency (ω_r). Our model includes a Matsuoka half-center oscillator for a central pattern generator (CPG) and a linear, one-degree-of-freedom system for a mechanical component. Using both an open-loop frequency response analysis and closed-loop simulations, we compared resonance tuning with four different feedback configurations as the mechanical resonant frequency, feedback gain, and mechanical damping varied. The feedback configurations consisted of two negative and two positive feedback connectivity schemes. We found that with negative feedback, resonance tuning predominantly occurred when ω_r was higher than the CPG’s endogenous frequency (ω_CPG). In contrast, with the two positive feedback configurations, resonance tuning only occurred if ω_r was lower than ω_CPG. Moreover, the differences in resonance tuning between the two positive (negative) feedback configurations increased with increasing feedback gain and with decreasing mechanical damping. Our results indicate that resonance tuning can be achieved with positive feedback. Furthermore, we have shown that the feedback configuration affects the parameter space over which the endogenous frequency of the CPG or resonant frequency the mechanical dynamics dominates the frequency of a rhythmic movement.     

Sensory Feedback Mechanism Underlying Entrainment of Central Pattern Generator to Mechanical Resonance

Rhythmic body motions observed in animal locomotion are known to be controlled by neuronal circuits called central pattern generators (CPGs). It appears that CPGs are energy efficient controllers that cooperate with biomechanical and environmental constraints through sensory feedback. In particular, the CPGs tend to induce rhythmic motion of the body at a natural frequency, i.e., the CPGs are entrained to a mechanical resonance by sensory feedback. The objective of this paper is to uncover the mechanism of entrainment resulting from the dynamic interaction of the CPG and mechanical system. We first develop multiple CPG models for the reciprocal inhibition oscillator (RIO) and examine through numerical experiments whether they can be entrained to a simple pendulum. This comparative study identifies the neuronal properties essential for the entrainment. We then analyze the simplest model that captures the essential dynamics via the method of harmonic balance. It is shown that robust entrainment results from a strong, positive-feedback coupling of a lightly damped mechanical system and the RIO consisting of neurons with the complete adaptation property     

Possible contributions of CPG activity to the control of rhythmic human arm movement

There is extensive modulation of cutaneous and H-reflexes during rhythmic leg movement in humans. Mechanisms controlling reflex modulation (e.g., phase- and task-dependent modulation, and reflex reversal) during leg movements have been ascribed to the activity of spinal central pattern generating (CPG) networks and peripheral feedback. Our working hypothesis has been that neural mechanisms (i.e., CPGs) controlling rhythmic movement are conserved between the human lumbar and cervical spinal cord. Thus reflex modulation during rhythmic arm movement should be similar to that for rhythmic leg movement. This hypothesis has been tested by studying the regulation of reflexes in arm muscles during rhythmic arm cycling and treadmill walking. This paper reviews recent studies that have revealed that reflexes in arm muscles show modulation within the movement cycle (e.g., phase-dependency and reflex reversal) and between static and rhythmic motor tasks (e.g., task-dependency). It is concluded that reflexes are modulated similarly during rhythmic movement of the upper and lower limbs, suggesting similar motor control mechanisms. One notable exception to this pattern is a failure of contralateral arm movement to modulate reflex amplitude, which contrasts directly with observations from the leg. Overall, the data support the hypothesis that CPG activity contributes to the neural control of rhythmic arm movement.     

Formal analysis of resonance entrainment by central pattern generator

The neuronal circuit controlling the rhythmic movements in animal locomotion is called the central pattern generator (CPG). The biological control mechanism appears to exploit mechanical resonance to achieve efficient locomotion. The objective of this paper is to reveal the fundamental mechanism underlying entrainment of CPGs to resonance through sensory feedback. To uncover the essential principle, we consider the simplest setting where a pendulum is driven by the reciprocal inhibition oscillator. Existence and properties of stable oscillations are examined by the harmonic balance method, which enables approximate but insightful analysis. In particular, analytical conditions are obtained under which harmonic balance predicts existence of an oscillation at a frequency near the resonance frequency. Our result reveals that the resonance entrainment can be maintained robustly against parameter perturbations through two distinct mechanisms: negative integral feedback and positive rate feedback.     

Associative neural network model for the generation of temporal patterns. Theory and application to central pattern generators.

Cyclic patterns of motor neuron activity are involved in the production of many rhythmic movements, such as walking, swimming, and scratching. These movements are controlled by neural circuits referred to as central pattern generators (CPGs). Some of these circuits function in the absence of both internal pacemakers and external feedback. We describe an associative neural network model whose dynamic behavior is similar to that of CPGs. The theory predicts the strength of all possible connections between pairs of neurons on the basis of the outputs of the CPG. It also allows the mean operating levels of the neurons to be deduced from the measured synaptic strengths between the pairs of neurons. We apply our theory to the CPG controlling escape swimming in the mollusk Tritonia diomedea. The basic rhythmic behavior is shown to be consistent with a simplified model that approximates neurons as threshold units and slow synaptic responses as elementary time delays. The model we describe may have relevance to other fixed action behaviors, as well as to the learning, recall, and recognition of temporally ordered information.     

The role of postinhibitory rebound in the locomotor central-pattern generator of Clione limacina

In animals, networks of central neurons, called central-patterngenerators (CPGs), produce a variety of locomotory behaviorsincluding walking, swimming, and flying. CPGs from diverse animalsshare many common characteristics that function at the systemlevel, circuit level, and cellular level. However, the relativeroles of common CPG characteristics are variable among differentanimal species, in ways that suit different forms of locomotionin different environmental contexts. Here, we examine some ofthese common features within the locomotor CPG in a model systemused to investigate changes in locomotory speed—the swimsystem of the pteropod mollusk, Clione limacina. In particular,we discuss the role of one cellular characteristic that is essentialfor locomotor pattern generation in Clione, postinhibitory rebound.     

Tuning the cochlea: wave-mediated positive feedback between cells

Frequency analysis by the mammalian cochlea is traditionally thought to occur via a hydrodynamically coupled ‘travelling wave’ along the basilar membrane. A persistent difficulty with this picture is how sharp tuning can emerge. This paper proposes, and models, a supplementary or alternative mechanism: it supposes that the cochlea analyses sound by setting up standing waves between parallel rows of outer hair cells. In this scheme, multiple cells mutually interact through positive feedback of wave-borne energy. Analytical modelling and numerical evaluation presented here demonstrate that this can provide narrow-band frequency analysis. Graded cochlear tuning will then rely on the distance between rows becoming greater as distance from the base increases (as exhibited by the actual cochlea) and on the wave’s phase velocity becoming slower. In effect, tuning is now a case of varying the feedback delay between the rows, and a prime candidate for a wave exhibiting suitably graded phase velocity—a short-wavelength ‘squirting wave’—is identified and used in the modelling. In this way, resonance between rows could supply both amplification and high Q, characteristics underlying the ‘cochlear amplifier’—the device whose action has long been evident to auditory science but whose anatomical basis and mode of operation are still obscure.     

Control of oscillation periods and phase durations in half-center central pattern generators: a comparative mechanistic analysis

Central pattern generators (CPGs) consisting of interacting groups of neurons drive a variety of repetitive, rhythmic behaviors in invertebrates and vertebrates, such as arise in locomotion, respiration, mastication, scratching, and so on. These CPGs are able to generate rhythmic activity in the absence of afferent feedback or rhythmic inputs. However, functionally relevant CPGs must adaptively respond to changing demands, manifested as changes in oscillation period or in relative phase durations in response to variations in non-patterned inputs or drives. Although many half-center CPG models, composed of symmetric units linked by reciprocal inhibition yet varying in their intrinsic cellular properties, have been proposed, the precise oscillatory mechanisms operating in most biological CPGs remain unknown. Using numerical simulations and phase-plane analysis, we comparatively investigated how the intrinsic cellular features incorporated in different CPG models, such as subthreshold activation based on a slowly inactivating persistent sodium current, adaptation based on slowly activating calcium-dependent potassium current, or post-inhibitory rebound excitation, can contribute to the control of oscillation period and phase durations in response to changes in excitatory external drive to one or both half-centers. Our analysis shows that both the sensitivity of oscillation period to alterations of excitatory drive and the degree to which the duration of each phase can be separately controlled depend strongly on the intrinsic cellular mechanisms involved in rhythm generation and phase transitions. In particular, the CPG formed from units incorporating a slowly inactivating persistent sodium current shows the greatest range of oscillation periods and the greatest degree of independence in phase duration control by asymmetric inputs. These results are explained based on geometric analysis of the phase plane structures corresponding to the dynamics for each CPG type, which in particular helps pinpoint the roles of escape and release from synaptic inhibition in the effects we find.     

Multi-neuronal refractory period adapts centrally generated behaviour to reward

Oscillating neuronal circuits, known as central pattern generators (CPGs), are responsible for generating rhythmic behaviours such as walking, breathing and chewing. The CPG model alone however does not account for the ability of animals to adapt their future behaviour to changes in the sensory environment that signal reward. Here, using multi-electrode array (MEA) recording in an established experimental model of centrally generated rhythmic behaviour we show that the feeding CPG of Lymnaea stagnalis is itself associated with another, and hitherto unidentified, oscillating neuronal population. This extra-CPG oscillator is characterised by high population-wide activity alternating with population-wide quiescence. During the quiescent periods the CPG is refractory to activation by food-associated stimuli. Furthermore, the duration of the refractory period predicts the timing of the next activation of the CPG, which may be minutes into the future. Rewarding food stimuli and dopamine accelerate the frequency of the extra-CPG oscillator and reduce the duration of its quiescent periods. These findings indicate that dopamine adapts future feeding behaviour to the availability of food by significantly reducing the refractory period of the brain's feeding circuitry.     

Sensory feedback in CNN-based central pattern generators

Central Pattern Generators (CPGs) are a suitable paradigm to solve the problem of locomotion control in walking robots. CPGs are able to generate feed-forward signals to achieve a proper coordination among the robot legs. In literature they are often modelled as networks of coupled nonlinear systems. However the topic of feedback in these systems is rarely addressed. On the other hand feedback is essential for locomotion. In this paper the CPG for a hexapod robot is implemented through Cellular Neural Networks (CNNs). Feedback is included in the CPG controller by exploiting the dynamic properties of the CPG motor-neurons, such as synchronization issue and local bifurcations. These universal paradigms provide the essential issues to include sensory feedback in CPG architectures based on coupled nonlinear systems. Experiments on a dynamic model of a hexapod robot are presented to validate the approach introduced.     

Beyond Muscles Stiffness: Importance of State-Estimation to Account for Very Fast Motor Corrections

Feedback delays are a major challenge for any controlled process, and yet we are able to easily control limb movements with speed and grace. A popular hypothesis suggests that the brain largely mitigates the impact of feedback delays (∼50 ms) by regulating the limb intrinsic visco-elastic properties (or impedance) with muscle co-contraction, which generates forces proportional to changes in joint angle and velocity with zero delay. Although attractive, this hypothesis is often based on estimates of limb impedance that include neural feedback, and therefore describe the entire motor system. In addition, this approach does not systematically take into account that muscles exhibit high intrinsic impedance only for small perturbations (short-range impedance). As a consequence, it remains unclear how the nervous system handles large perturbations, as well as disturbances encountered during movement when short-range impedance cannot contribute. We address this issue by comparing feedback responses to load pulses applied to the elbow of human subjects with theoretical simulations. After validating the model parameters, we show that the ability of humans to generate fast and accurate corrective movements is compatible with a control strategy based on state estimation. We also highlight the merits of delay-uncompensated robust control, which can mitigate the impact of internal model errors, but at the cost of slowing feedback corrections. We speculate that the puzzling observation of presynaptic inhibition of peripheral afferents in the spinal cord at movement onset helps to counter the destabilizing transition from high muscle impedance during posture to low muscle impedance during movement.     

Neural circuit flexibility in a small sensorimotor system

Neuronal circuits underlying rhythmic behaviors (central pattern generators: CPGs) can generate rhythmic motor output without sensory input. However, sensory input is pivotal for generating behaviorally relevant CPG output. Here we discuss recent work in the decapod crustacean stomatogastric nervous system (STNS) identifying cellular and synaptic mechanisms whereby sensory inputs select particular motor outputs from CPG circuits. This includes several examples in which sensory neurons regulate the impact of descending projection neurons on CPG circuits. This level of analysis is possible in the STNS due to the relatively unique access to identified circuit, projection, and sensory neurons. These studies are also revealing additional degrees of freedom in sensorimotor integration that underlie the extensive flexibility intrinsic to rhythmic motor systems.     

Dynamic head-neck stabilization and modulation with perturbation bandwidth investigated using a multisegment neuromuscular model

The human head-neck system requires continuous stabilization in the presence of gravity and trunk motion. We investigated contributions of the vestibulocollic reflex (VCR), the cervicocollic reflex (CCR), and neck muscle co-contraction to head-in-space and head-on-trunk stabilization, and investigated modulation of the stabilization strategy with the frequency content of trunk perturbations and the presence of visual feedback.We developed a multisegment cervical spine model where reflex gains (VCR and CCR) and neck muscle co-contraction were estimated by fitting the model to the response of young healthy subjects, seated and exposed to anterior-posterior trunk motion, with frequency content from 0.3 up to 1, 2, 4 and 8 Hz, with and without visual feedback.The VCR contributed to head-in-space stabilization with a strong reduction of head rotation (<8 Hz) and a moderate reduction of head translation (>1 Hz). The CCR contributed to head-on-trunk stabilization with a reduction of head rotation and head translation relative to the trunk (<2 Hz). The CCR also proved essential to stabilize the individual intervertebral joints and prevent neck buckling. Co-contraction was estimated to be of minor relevance. Control strategies employed during low bandwidth perturbations most effectively reduced head rotation and head relative displacement up to 3 Hz while control strategies employed during high bandwidth perturbations reduced head global translation between 1 and 4 Hz. This indicates a shift from minimizing head-on-trunk rotation and translation during low bandwidth perturbations to minimizing head-in-space translation during high bandwidth perturbations. Presence of visual feedback had limited effects suggesting increased usage of vestibular feedback.     

Modeling movement disorders--CRPS-related dystonia explained by abnormal proprioceptive reflexes

Humans control their movements using adaptive proprioceptive feedback from muscle afferents. The interaction between proprioceptive reflexes and biomechanical properties of the limb is essential in understanding the etiology of movement disorders. A non-linear neuromuscular model of the wrist incorporating muscle dynamics and neural control was developed to test hypotheses on fixed dystonia. Dystonia entails sustained muscle contractions resulting in abnormal postures. Lack of inhibition is often hypothesized to result in hyperreflexia (exaggerated reflexes), which may cause fixed dystonia. In this study the model-simulated behavior in case of several abnormal reflex settings was compared to the clinical features of dystonia: abnormal posture, sustained muscle contraction, increased stiffness, diminished voluntary control and activity-aggravation. The simulation results were rated to criteria based on characteristic features of dystonia. Three abnormal reflex scenarios were tested: (1) increased reflex sensitivity-increased sensitivity of both the agonistic and antagonistic reflex pathways; (2) imbalanced reflex offset-a static offset to the reflex pathways on the agonistic side only; and (3) imbalanced reflex sensitivity-increased sensitivity of only the agonistic reflex pathways. Increased reflex sensitivity did not fully account for the features of dystonia, despite distinct motor dysfunction, since no abnormal postures occurred. Although imbalanced reflex offset did result in an abnormal posture, it could not satisfy other criteria. Nevertheless, imbalanced reflex sensitivity with unstable force feedback in one of the antagonists closely resembled all features of dystonia. The developed neuromuscular model is an effective tool to test hypotheses on the underlying pathophysiology of movement disorders.     

Partly shared spinal cord networks for locomotion and scratching

Animals produce a variety of behaviors using a limited number of muscles and motor neurons. Rhythmic behaviors are often generated in basic form by networks of neurons within the central nervous system, or central pattern generators (CPGs). It is known from several invertebrates that different rhythmic behaviors involving the same muscles and motor neurons can be generated by a single CPG, multiple separate CPGs, or partly overlapping CPGs. Much less is known about how vertebrates generate multiple, rhythmic behaviors involving the same muscles. The spinal cord of limbed vertebrates contains CPGs for locomotion and multiple forms of scratching. We investigated the extent of sharing of CPGs for hind limb locomotion and for scratching. We used the spinal cord of adult red-eared turtles. Animals were immobilized to remove movement-related sensory feedback and were spinally transected to remove input from the brain. We took two approaches. First, we monitored individual spinal cord interneurons (i.e., neurons that are in between sensory neurons and motor neurons) during generation of each kind of rhythmic output of motor neurons (i.e., each motor pattern). Many spinal cord interneurons were rhythmically activated during the motor patterns for forward swimming and all three forms of scratching. Some of these scratch/swim interneurons had physiological and morphological properties consistent with their playing a role in the generation of motor patterns for all of these rhythmic behaviors. Other spinal cord interneurons, however, were rhythmically activated during scratching motor patterns but inhibited during swimming motor patterns. Thus, locomotion and scratching may be generated by partly shared spinal cord CPGs. Second, we delivered swim-evoking and scratch-evoking stimuli simultaneously and monitored the resulting motor patterns. Simultaneous stimulation could cause interactions of scratch inputs with subthreshold swim inputs to produce normal swimming, acceleration of the swimming rhythm, scratch-swim hybrid cycles, or complete cessation of the rhythm. The type of effect obtained depended on the level of swim-evoking stimulation. These effects suggest that swim-evoking and scratch-evoking inputs can interact strongly in the spinal cord to modify the rhythm and pattern of motor output. Collectively, the single-neuron recordings and the results of simultaneous stimulation suggest that important elements of the generation of rhythms and patterns are shared between locomotion and scratching in limbed vertebrates.     

Central pattern generators: some principles learned from invertebrate model systems.

1. Central pattern generators (CPGs) underlie a wide variety of rhythmic behaviours such as locomotion and respiration in most multi-cellular organisms. 2. The CPG's are capable of generating a patterned output without phasic sensory input. 3. The organization of the CPG is due to both intrinsic properties of the individual neurons and their network interactions. 4. To gain an understanding of the mechanisms which underlie rhythmicity a CPG has been reconstructed in culture. This will allow investigators to test directly the mechanisms underlying the generation of rhythmic output and will allow the direct testing of the mechanisms by which various modulators affect the CPG.     

Convergence in Reflex Pathways from Multiple Cutaneous Nerves Innervating the Foot Depends upon the Number of Rhythmically Active Limbs during Locomotion

Neural output from the locomotor system for each arm and leg influences the spinal motoneuronal pools directly and indirectly through interneuronal (IN) reflex networks. While well documented in other species, less is known about the functions and features of convergence in common IN reflex system from cutaneous afferents innervating different foot regions during remote arm and leg movement in humans. The purpose of the present study was to use spatial facilitation to examine possible convergence in common reflex pathways during rhythmic locomotor limb movements. Cutaneous reflexes were evoked in ipsilateral tibialis anterior muscle by stimulating (in random order) the sural nerve (SUR), the distal tibial nerve (TIB), and combined simultaneous stimulation of both nerves (TIB&SUR). Reflexes were evoked while participants performed rhythmic stepping and arm swinging movement with both arms and the leg contralateral to stimulation (ARM&LEG), with just arm movement (ARM) and with just contralateral leg movement (LEG). Stimulation intensities were just below threshold for evoking early latency (<80 ms to peak) reflexes. For each stimulus condition, rectified EMG signals were averaged while participants held static contractions in the stationary (stimulated) leg. During ARM&LEG movement, amplitudes of cutaneous reflexes evoked by combined TIB&SUR stimulation were significantly larger than simple mathematical summation of the amplitudes evoked by SUR or TIB alone. Interestingly, this extra facilitation seen during combined nerve stimulation was significantly reduced when performing ARM or LEG compared to ARM&LEG. We conclude that locomotor rhythmic limb movement induces excitation of common IN reflex pathways from cutaneous afferents innervating different foot regions. Importantly, activity in this pathway is most facilitated during ARM&LEG movement. These results suggest that transmission in IN reflex pathways is weighted according to the number of limbs directly engaged in human locomotor activity and underscores the importance of arm swing to support neuronal excitability in leg muscles.     

A mathematical modeling study of inter-segmental coordination during stick insect walking

The biomechanical conditions for walking in the stick insect require a modeling approach that is based on the control of pairs of antagonistic motoneuron (MN) pools for each leg joint by independent central pattern generators (CPGs). Each CPG controls a pair of antagonistic MN pools. Furthermore, specific sensory feedback signals play an important role in the control of single leg movement and in the generation of inter-leg coordination or the interplay between both tasks. Currently, however, no mathematical model exists that provides a theoretical approach to understanding the generation of coordinated locomotion in such a multi-legged locomotor system. In the present study, I created such a theoretical model for the stick insect walking system, which describes the MN activity of a single forward stepping middle leg and helps to explain the neuronal mechanisms underlying coordinating information transfer between ipsilateral legs. In this model, CPGs that belong to the same leg, as well as those belonging to different legs, are connected by specific sensory feedback pathways that convey information about movements and forces generated during locomotion. The model emphasizes the importance of sensory feedback, which is used by the central nervous system to enhance weak excitatory and inhibitory synaptic connections from front to rear between the three thorax-coxa-joint CPGs. Thereby the sensory feedback activates caudal pattern generation networks and helps to coordinate leg movements by generating in-phase and out-of-phase thoracic MN activity.     

Feedback control of the limbs position during voluntary rhythmic oscillation

The mechanisms that control the limbs position during rhythmic voluntary oscillations were investigated in ten subjects, who were asked to synchronise the lower peak of their hand or foot rhythmic oscillations to a metronome beat. The efficacy of the “position control” was estimated by measuring the degree of synchronisation between the metronome signal and the requested limb position and how it was affected by changing both the oscillation frequency (between 0.4 and 3.0 Hz) and the limbs inertial properties. With the limbs unloaded, the lower peak of both the hand and foot oscillations lagged the metronome beat of a slight amount that remained constant over the whole frequency range (mean phase delay −13.2° for the hand and −4.7° for the foot). The constancy was obtained by phase-advancing, at each frequency increment, the electromyogram (EMG) activation with respect of the clock beat of the amount necessary to compensate for the simultaneous increase of the lag between the EMG and the movement, produced by the limb mechanical impedance. After loading of either limb, the increase of the oscillation frequency induced larger EMG-movement delays and the anticipatory compensation became insufficient, so that the movement progressively phase-lagged the clock beat. The above results have been accurately simulated by a neural network connected to a pendulum model that shared the same mechanical properties of the moving limb. The network compares a central command (the intended position) to the actual position of the effector and acts as a closed-loop proportional, integrative and derivative controller. It is proposed that the synchronisation of rhythmic oscillations of either the hand or the foot is sustained by a feed-back control that conforms the position of each limb to that encoded in the central voluntary command.