Multi-layered multi-pattern CPG for adaptive locomotion of humanoid robots

In this paper, we present an extended mathematical model of the central pattern generator (CPG) in the spinal cord. The proposed CPG model is used as the underlying low-level controller of a humanoid robot to generate various walking patterns. Such biological mechanisms have been demonstrated to be robust in locomotion of animal. Our model is supported by two neurophysiological studies. The first study identified a neural circuitry consisting of a two-layered CPG, in which pattern formation and rhythm generation are produced at different levels. The second study focused on a specific neural model that can generate different patterns, including oscillation. This neural model was employed in the pattern generation layer of our CPG, which enables it to produce different motion patterns—rhythmic as well as non-rhythmic motions. Due to the pattern-formation layer, the CPG is able to produce behaviors related to the dominating rhythm (extension/flexion) and rhythm deletion without rhythm resetting. The proposed multi-layered multi-pattern CPG model (MLMP-CPG) has been deployed in a 3D humanoid robot (NAO) while it performs locomotion tasks. The effectiveness of our model is demonstrated in simulations and through experimental results.     

Computer simulation study on central pattern generator: from biology to engineering

Central pattern generator (CPG) is a neuronal circuit in the nervous system that can generate oscillatory patterns for the rhythmic movements. Its simplified format, neural oscillator, is wildly adopted in engineering application. This paper explores the CPG from an integral view that combines biology and engineering together. Biological CPG and simplified CPG are both studied. Computer simulation reveals the mechanism of CPG. Some properties, such as effect of tonic input and sensory feedback, stable oscillation, robustness, entrainment etc., are further studied. The promising results provide foundation for the potential engineering application in future.     

Rhythmic behaviour and pattern-generating circuits in the locust: Key concepts and recent updates

There is growing recognition that rhythmic activity patterns are widespread in our brain and play an important role in all aspects of the functioning of our nervous system, from sensory integration to central processing and motor control. The study of the unique properties that enable central circuits to generate their rhythmic output in the absence of any patterned, sensory or descending, inputs, has been very rewarding in the relatively simple invertebrate preparations. The locust, specifically, is a remarkable example of an organism in which central pattern generator (CPG) networks have been suggested and studied in practically all aspects of their behaviour. Here we present an updated overview of the various rhythmic behaviours in the locust and aspects of their neural control. We focus on the fundamental concepts of multifunctional neuronal circuits, neural centre interactions and neuromodulation of CPG networks. We are certain that the very broad and solid knowledge base of locust rhythmic behaviour and pattern-generating circuits will continue to expand and further contribute to our understanding of the principles behind the functioning of the nervous system and, indeed, the brain.     

Transmitter-dependent involvement of the respiratory interneurons in the locomotor rhythm in the pulmonate mollusc Lymnaea

Pedal cell RPeD1 of the pond snail L. stagnalis becomes involved in a central rhythm identified as an activity of the central pattern generator (CPG) for locomotion. The RPeD1 rhythm developed as driven by a synaptic input in isolated CNS preparations treated with 0.05 mM serotonin (5HT) or 0.1 mM 5-hydroxytryptophan (5HTP). The 5HT-induced co-ordinated rhythmic activity was retained by each of two pedal ganglia after complete isolation thus suggesting that the respective CPG lies entirely within the pedal portion of the CNS and is paired. The findings suggest that the RPeD1 switching from one network to another represents a neurotransmitter-dependent phenomenon.     

The contribution of sensory inputs to the pattern generation of breathing

Current concepts of the basic neural control system and its modulation by afferent inputs are reviewed. It is emphasized that, in analogy with locomotion, the central pattern generator (CPG) for automatic metabolic respiration does not depend on any afferent feedback from receptors sensitive to the movements of the "pump," or the streams of pumped air, for its production of a rhythmic motor output provides the CPG receives some "drive" inputs above threshold and adequate bias. The operation for a variety of reflexes and feedback loops is of fundamental importance, however, for adapting the breathing pattern to the varying requirements for gas exchange to the many behavioural, nonmetabolic demands on the breathing apparatus which are competing with its primary metabolic control functions. The presentation is focussed also on available evidence that the respiratory CPG exerts powerful modulations on the transmission in these reflex pathways controlling the pattern of breathing and adjusting it to the various metabolic and behavioural demands. Mechanisms for "gating," "phasic gain changes," and "phase-dependent reflex reversal" are exemplified.     

Loss of self-inhibition is a cellular mechanism for episodic rhythmic behavior

BACKGROUND: Rhythmic motor behaviors can be generated continuously (e.g., breathing) or episodically (e.g., locomotion, swallowing), when short or long bouts of rhythmic activity are interspersed with periods of quiescence. Although the mechanisms of rhythm generation are known in detail in many systems, there is very little understanding of how the episodic nature of rhythmic behavior is produced at the neuronal level. RESULTS: Using a well-established episodic rhythm-generating neural circuit controlling molluscan feeding, we demonstrate that quiescence between bouts of activity arises from active, maintained inhibition of an otherwise rhythmically active network. We show that the source of the suppressive drive is within the circuit itself; a single central pattern generator (CPG) interneuron type that fires tonically to inhibit feeding during quiescence. Suppression of the tonic activity of this neuron by food is sufficient to change the network from an inactive to a rhythmically active state, with the cell switching function to fire phasically as part of the food-evoked rhythmogenesis. Furthermore, the absolute level of intrinsic suppressive control is modulated extrinsically by the animal's behavioral state (e.g., hunger/satiety), increasing the probability of episodes of feeding when the animal is hungry. CONCLUSIONS: By utilizing the same intrinsic member of a CPG network in both rhythm-generation and suppression, this system has developed a simple and efficient mechanism for generating a variable level of response to suit the animal's changing behavioral demands.     

Neuromodulation of central pattern generators in invertebrates and vertebrates

Central pattern generators are subject to extensive modulation that generates flexibility in the rhythmic outputs of these neural networks. The effects of neuromodulators interact with one another, and modulatory neurons are themselves often subject to modulation, enabling both higher order control and indirect interactions among central pattern generators. In addition, modulators often directly mediate the interactions between functionally related central pattern generators. In systems such as the vertebrate respiratory central pattern generator, multiple pacemaker types interact to produce rhythmic output. Modulators can then alter the relative contributions of the different pacemakers, leading to substantial changes in motor output and hence to different behaviors. Surprisingly, substantial changes in some aspects of the circuitry of a central pattern generator, such as a several-fold increase in synaptic strength, can sometimes have little effect on the output of the CPG, whereas other changes have profound effects.     

Sprouting and connectivity of embryonic leech heart excitor (HE) motor neurons in the absence of their peripheral target

The rhythmic pumping of the hearts in the medicinal leech,Hirudo medicinalis, is neurogenic and mediated by a defined circuit involving identified interneurons in a central pattern generator (CPG) and segmentally iterated motor neurons that drive the heart muscle. During early embryogenesis, presumptive heart excitor (HE) motor neurons extend many axon branches into the body wall; they later innervate the heart while retracting the supernumerary peripheral axons, and only much later in development receive synaptic input from the central pattern generator (Jellies, Kopp and Bledsoe (1992)J. Exp. Biol., 170, 71–92.)- In this study, HE motor neurons were deprived of an early interaction with the heart by surgical ablation of a circumscribed portion of body wall including the heart primordium. Anatomical and electrophysiological data were obtained using intracellular techniques to examine the hypothesis that peripheral interactions with the developing heart provide instructive cues for the final differentiation of these neurons. Target-deprived HE motor neurons continued to extend multiple axons in ventral, lateral and dorsal body wall throughout late embryonic and into postembryonic stages and they extended anomalous axons within the CNS. This resembles the early embryonic growth of HE motor neurons before heart tube differentiation. Furthermore, HE motor neurons deprived of heart contact exhibited tonic activity similar to the situation during early development before they are contacted by the CPG interneurons. In contrast, sham-operated and contralateral HE motor neurons oscillated normally. These results suggest that heart tube contact is specifically required for at least some aspects of HE development and provide a framework in which to identify cell-cell interactions that are involved in matching neurons and targets to generate behaviorally relevant neural circuits.     

Serotonin regulates rhythmic whisking

Many rodents explore their environment by rhythmically palpating objects with their mystacial whiskers. These rhythmic whisker movements ("whisking"; 5-9 Hz) are thought to be regulated by an unknown brainstem central pattern generator (CPG). We tested the hypothesis that serotonin (5-HT) inputs to whisking facial motoneurons (wFMNs) are part of this CPG. In response to exogenous serotonin, wFMNs recorded in vitro fire rhythmically at whisking frequencies, and selective 5-HT2 or 5-HT3 receptor antagonists suppress this rhythmic firing. In vivo, stimulation of brainstem serotonergic raphe nuclei evokes whisker movements. Unilateral infusion of selective 5-HT2 or 5-HT3 receptor antagonists suppresses ipsilateral whisking and substantially alters the frequencies and symmetry of whisker movements. These findings suggest that serotonin is both necessary and sufficient to generate rhythmic whisker movements and that serotonergic premotoneurons are part of a whisking CPG.     

Possible contributions of CPG activity to the control of rhythmic human arm movement

There is extensive modulation of cutaneous and H-reflexes during rhythmic leg movement in humans. Mechanisms controlling reflex modulation (e.g., phase- and task-dependent modulation, and reflex reversal) during leg movements have been ascribed to the activity of spinal central pattern generating (CPG) networks and peripheral feedback. Our working hypothesis has been that neural mechanisms (i.e., CPGs) controlling rhythmic movement are conserved between the human lumbar and cervical spinal cord. Thus reflex modulation during rhythmic arm movement should be similar to that for rhythmic leg movement. This hypothesis has been tested by studying the regulation of reflexes in arm muscles during rhythmic arm cycling and treadmill walking. This paper reviews recent studies that have revealed that reflexes in arm muscles show modulation within the movement cycle (e.g., phase-dependency and reflex reversal) and between static and rhythmic motor tasks (e.g., task-dependency). It is concluded that reflexes are modulated similarly during rhythmic movement of the upper and lower limbs, suggesting similar motor control mechanisms. One notable exception to this pattern is a failure of contralateral arm movement to modulate reflex amplitude, which contrasts directly with observations from the leg. Overall, the data support the hypothesis that CPG activity contributes to the neural control of rhythmic arm movement.     

A mathematical model of adaptive behavior in quadruped locomotion

Locomotion involves repetitive movements and is often executed unconsciously and automatically. In order to achieve smooth locomotion, the coordination of the rhythms of all physical parts is important. Neurophysiological studies have revealed that basic rhythms are produced in the spinal network called, the central pattern generator (CPG), where some neural oscillators interact to self-organize coordinated rhythms. We present a model of the adaptation of locomotion patterns to a variable environment, and attempt to elucidate how the dynamics of locomotion pattern generation are adjusted by the environmental changes. Recent experimental results indicate that decerebrate cats have the ability to learn new gait patterns in a changed environment. In those experiments, a decerebrate cat was set on a treadmill consisting of three moving belts. This treadmill provides a periodic perturbation to each limb through variation of the speed of each belt. When the belt for the left forelimb is quickened, the decerebrate cat initially loses interlimb coordination and stability, but gradually recovers them and finally walks with a new gait. Based on the above biological facts, we propose a CPG model whose rhythmic pattern adapts to periodic perturbation from the variable environment. First, we design the oscillator interactions to generate a desired rhythmic pattern. In our model, oscillator interactions are regarded as the forces that generate the desired motion pattern. If the desired pattern has already been realized, then the interactions are equal to zero. However, this rhythmic pattern is not reproducible when there is an environmental change. Also, if we do not adjust the rhythmic dynamics, the oscillator interactions will not be zero. Therefore, in our adaptation rule, we adjust the memorized rhythmic pattern so as to minimize the oscillator interactions. This rule can describe the adaptive behavior of decerebrate cats well. Finally, we propose a mathematical framework of an adaptation in rhythmic motion. Our framework consists of three types of dynamics: environmental, rhythmic motion, and adaptation dynamics. We conclude that the time scale of adaptation dynamics should be much larger than that of rhythmic motion dynamics, and the repetition of rhythmic motions in a stable environment is important for the convergence of adaptation. Received: 10 July 1997 / Accepted in revised form: 13 March 1998     

Temperature coupling in cricket acoustic communication

Acoustic communication in Gryllus firmus is temperature-coupled: temperature induces parallel changes in male calling song temporal pattern, and in female preference for song. Temperature effects on song production and recognition networks were localized by selectively warming head or thorax or both head and thorax of intact crickets, then eliciting aggression song production (males) or phonotaxis to synthetic calling song (females). Because male song is produced by a thoracic central pattern generator (CPG), and because head ganglia are necessary for female song recognition, measurements of female phonotaxis under such conditions may be used to test the following competing hypotheses about organization of the song recognition network: 1. A set of neurons homologous to the male song CPG exist in the female, and are used as a template that determines preferred values of song temporal parameters for song pattern recognition (the common neural elements hypothesis), and 2. temporal pattern preference is determined entirely within the head ganglia. Neither selective warming of the head nor of the thorax was effective in changing female song preference, but simultaneous warming of head and thorax shifted preference toward a faster song in most preparations, as did warming the whole animal by raising ambient temperature. These results suggest that phonotactic preference for song temporal pattern is plurisegmentally determined in field crickets. Selective warming experiments during aggression song production in males revealed that syllable period is influenced but not completely determined by thoracic temperature; head temperature is irrelevant. The song CPG appears to receive some rate-setting information from outside the thoracic central nervous system.     

Neuromodulation and flexibility in Central Pattern Generator networks

Central Pattern Generator (CPG) networks, which organize rhythmic movements, have long served as models for neural network organization. Modulatory inputs are essential components of CPG function: neuromodulators set the parameters of CPG neurons and synapses to render the networks functional. Each modulator acts on the network by many effects which may oppose one another; this may serve to stabilize the modulated state. Neuromodulators also determine the active neuronal composition in the CPG, which varies with state changes such as locomotor speed. The pattern of gene expression which determines the electrophysiological personality of each CPG neuron is also under modulatory control. It is not possible to model the function of neural networks without including the actions of neuromodulators.     

Hormone‐mediated modulation of the electromotor CPG in pulse‐type weakly electric fish. Commonalities and differences across species

Like stomatogastric activity in crustaceans, vocalization in teleosts and frogs, and locomotion in mammals, the electric organ discharge (EOD) of weakly electric fish is a rhythmic and stereotyped electromotor pattern. The EOD, which functions in both perception and communication, is controlled by a two‐layered central pattern generator (CPG), the electromotor CPG, which modifies its basal output in response to environmental and social challenges. Despite major anatomo‐functional commonalities in the electromotor CPG across electric fish species, we show that Gymnotus omarorum and Brachyhypopomus gauderio have evolved divergent neural processes to transiently modify the CPG outputs through descending fast neurotransmitter inputs to generate communication signals. We also present two examples of electric behavioral displays in which it is possible to separately analyze the effects of neuropeptides (mid‐term modulation) and gonadal steroid hormones (long‐term modulation) upon the CPG. First, the nonbreeding territorial aggression of G. omarorum has been an advantageous model to analyze the status‐dependent modulation of the excitability of CPG neuronal components by vasotocin. Second, the seasonal and sexually dimorphic courtship signals of B. gauderio have been useful to understand the effects of sex steroids on the responses to glutamatergic inputs in the CPG. Overall, the electromotor CPG functions in a regime that safeguards the EOD waveform. However, prepacemaker influences and hormonal modulation enable an enormous versatility and allows the EOD to adapt its functional state in a species‐, sex‐, and social context‐specific manners.     

Multi-neuronal refractory period adapts centrally generated behaviour to reward

Oscillating neuronal circuits, known as central pattern generators (CPGs), are responsible for generating rhythmic behaviours such as walking, breathing and chewing. The CPG model alone however does not account for the ability of animals to adapt their future behaviour to changes in the sensory environment that signal reward. Here, using multi-electrode array (MEA) recording in an established experimental model of centrally generated rhythmic behaviour we show that the feeding CPG of Lymnaea stagnalis is itself associated with another, and hitherto unidentified, oscillating neuronal population. This extra-CPG oscillator is characterised by high population-wide activity alternating with population-wide quiescence. During the quiescent periods the CPG is refractory to activation by food-associated stimuli. Furthermore, the duration of the refractory period predicts the timing of the next activation of the CPG, which may be minutes into the future. Rewarding food stimuli and dopamine accelerate the frequency of the extra-CPG oscillator and reduce the duration of its quiescent periods. These findings indicate that dopamine adapts future feeding behaviour to the availability of food by significantly reducing the refractory period of the brain's feeding circuitry.     

Invertebrate central pattern generator circuits

There are now a reasonable number of invertebrate central pattern generator (CPG) circuits described in sufficient detail that a mechanistic explanation of how they work is possible. These small circuits represent the best-understood neural circuits with which to investigate how cell-to-cell synaptic connections and individual channel conductances combine to generate rhythmic and patterned output. In this review, some of the main lessons that have appeared from this analysis are discussed and concrete examples of circuits ranging from single phase to multiple phase patterns are described. While it is clear that the cellular components of any CPG are basically the same, the topology of the circuits have evolved independently to meet the particular motor requirements of each individual organism and only a few general principles of circuit operation have emerged. The principal usefulness of small systems in relation to the brain is to demonstrate in detail how cellular infrastructure can be used to generate rhythmicity and form specialized patterns in a way that may suggest how similar processes might occur in more complex systems. But some of the problems and challenges associated with applying data from invertebrate preparations to the brain are also discussed. Finally, I discuss why it is useful to have well-defined circuits with which to examine various computational models that can be validated experimentally and possibly applied to brain circuits when the details of such circuits become available.     

Neuromodulation for behavior in the locust frontal ganglion

Neuromodulators orchestrate complex behavioral routines by their multiple and combined effects on the nervous system. In the desert locust, Schistocerca gregaria, frontal ganglion neurons innervate foregut dilator muscles and play a key role in the control of foregut motor patterns. To further investigate the role of the frontal ganglion in locust behavior, we currently focus on the frontal ganglion central pattern generator as a target for neuromodulation. Application of octopamine, a well-studied insect neuromodulator, generated reversible disruption of frontal ganglion rhythmic activity. The threshold for the modulatory effects of octopamine was 10^–6 mol l^–1, and 10^–4 mol l^–1 always abolished the ongoing rhythm. In contrast to this straightforward modulation, allatostatin, previously reported to be a myoinhibitor of insect gut muscles, showed complex, tri-modal, dose-dependent effects on frontal ganglion rhythmic pattern. Using a novel cross-correlation analysis technique, we show that different allatostatin concentrations have very different effects not only on cycle period but also on temporal characteristics of the rhythmic bursts of action potentials. Allatostatin also altered the frontal ganglion rhythm in vivo. The analysis technique we introduce may be instrumental in the study of not fully characterized neural circuits and their modulation. The physiological significance of our results and the role of the modulators in locust behavior are discussed.Abbreviation CPG central pattern generator - FG frontal ganglion - JH juvenile hormone - STNS stomatogastric nervous system     

A central pattern generator underlies crawling in the medicinal leech

Crawling in the medicinal leech has previously been thought to require sensory feedback because the intact behavior is strongly modulated by sensory feedback and because semi-intact preparations will only crawl if they can move freely. Here we show that an isolated leech nerve cord can produce a crawling motor pattern similar to the one seen in semi-intact preparations, which consists of an anterior-to-posterior wave of alternating excitatory circular and longitudinal motor neuron bursts in each segment. The isolated cord also reproduces the patterns of activity seen in semi-intact preparations for several other kinds of cells: the dorsal inhibitor cell 1, the ventral excitor cell 4, and the annulus erector motor neuron. Because this correspondence is so strong, there must be a central pattern generator in the isolated cord that can produce the basic motor pattern for crawling without sensory feedback. A quantitative analysis of the isolated motor pattern, however, reveals that isolated and semi-intact preparations have longer periods than the intact behavior and that there are deficiencies in the timing of motor neuron bursts in the isolated pattern. These results suggest that sensory feedback modulates the isolated central pattern generator to help produce the normal motor pattern.     

Systems-level modeling of neuronal circuits for leech swimming

This paper describes a mathematical model of the neuronal central pattern generator (CPG) that controls the rhythmic body motion of the swimming leech. The systems approach is employed to capture the neuronal dynamics essential for generating coordinated oscillations of cell membrane potentials by a simple CPG architecture with a minimal number of parameters. Based on input/output data from physiological experiments, dynamical components (neurons and synaptic interactions) are first modeled individually and then integrated into a chain of nonlinear oscillators to form a CPG. We show through numerical simulations that the values of a few parameters can be estimated within physiologically reasonable ranges to achieve good fit of the data with respect to the phase, amplitude, and period. This parameter estimation leads to predictions regarding the synaptic coupling strength and intrinsic period gradient along the nerve cord, the latter of which agrees qualitatively with experimental observations.