A mathematical model of adaptive behavior in quadruped locomotion

Locomotion involves repetitive movements and is often executed unconsciously and automatically. In order to achieve smooth locomotion, the coordination of the rhythms of all physical parts is important. Neurophysiological studies have revealed that basic rhythms are produced in the spinal network called, the central pattern generator (CPG), where some neural oscillators interact to self-organize coordinated rhythms. We present a model of the adaptation of locomotion patterns to a variable environment, and attempt to elucidate how the dynamics of locomotion pattern generation are adjusted by the environmental changes. Recent experimental results indicate that decerebrate cats have the ability to learn new gait patterns in a changed environment. In those experiments, a decerebrate cat was set on a treadmill consisting of three moving belts. This treadmill provides a periodic perturbation to each limb through variation of the speed of each belt. When the belt for the left forelimb is quickened, the decerebrate cat initially loses interlimb coordination and stability, but gradually recovers them and finally walks with a new gait. Based on the above biological facts, we propose a CPG model whose rhythmic pattern adapts to periodic perturbation from the variable environment. First, we design the oscillator interactions to generate a desired rhythmic pattern. In our model, oscillator interactions are regarded as the forces that generate the desired motion pattern. If the desired pattern has already been realized, then the interactions are equal to zero. However, this rhythmic pattern is not reproducible when there is an environmental change. Also, if we do not adjust the rhythmic dynamics, the oscillator interactions will not be zero. Therefore, in our adaptation rule, we adjust the memorized rhythmic pattern so as to minimize the oscillator interactions. This rule can describe the adaptive behavior of decerebrate cats well. Finally, we propose a mathematical framework of an adaptation in rhythmic motion. Our framework consists of three types of dynamics: environmental, rhythmic motion, and adaptation dynamics. We conclude that the time scale of adaptation dynamics should be much larger than that of rhythmic motion dynamics, and the repetition of rhythmic motions in a stable environment is important for the convergence of adaptation. Received: 10 July 1997 / Accepted in revised form: 13 March 1998     

The neural circuits and synaptic mechanisms underlying motor initiation in C. elegans

C. elegans is widely used to dissect how neural circuits and genes generate behavior. During locomotion, worms initiate backward movement to change locomotion direction spontaneously or in response to sensory cues; however, the underlying neural circuits are not well defined. We applied a multidisciplinary approach to map neural circuits in freely behaving worms by integrating functional imaging, optogenetic interrogation, genetic manipulation, laser ablation, and electrophysiology. We found that a disinhibitory circuit and a stimulatory circuit together promote initiation of backward movement and that circuitry dynamics is differentially regulated by sensory cues. Both circuits require glutamatergic transmission but depend on distinct glutamate receptors. This dual mode of motor initiation control is found in mammals, suggesting that distantly related organisms with anatomically distinct nervous systems may adopt similar strategies for motor control. Additionally, our studies illustrate how a multidisciplinary approach facilitates dissection of circuit and synaptic mechanisms underlying behavior in a genetic model organism.     

Traveling-wave pattern generator controls movement and organization of sensory feedback in a spinal cord model

 A traveling wave in a two-dimensional spinal cord model constitutes a stable pattern generator for quadruped gaits. In the context of the somatotopic organization of the spinal cord, this pattern generator is sufficient to generate stable locomotive limb trajectories. The elastic properties of muscles alone, providing linear negative feedback, are sufficient to stabilize stance and locomotion in the presence of perturbative forces. We further show that such a pattern generator is capable of organizing sensory processing in the spinal cord. A single-layer perceptron was trained to associate the sensory feedback from the limb (coding force, length, and change of length for each muscle) with the two-dimensional activity profile of the traveling wave. This resulted in a well-defined spatial organization of the connections within the spinal network along a rostrocaudal axis. The spinal network driven by peripheral afferents alone supported autonomous locomotion in the positive feedback mode, whereas in the negative feedback mode stance was stabilized in response to perturbations. Systematic variation of a parameter representing the effect of gamma-motor neurons on muscle spindle activity in our model led to a corresponding shift of limb position during stance and locomotion, resulting in a systematic displacement alteration of foot positions. Received: 30 July 2001 / Accepted in revised form: 17 April 2002 Correspondence to: A. Kaske (e-mails: alexander.kaske@mtc.ki.se, alexander.kaske@vglab.com)     

Mechanical models for insect locomotion: dynamics and stability in the horizontal plane I. Theory

We study the dynamics and stability of legged locomotion in the horizontal plane. Motivated by experimental studies of insects, we develop two- and three-degree-of freedom rigid body models with pairs of ‘virtual’ elastic legs in intermittent contact with the ground. We focus on conservative compliant-legged models, but we also consider prescribed forces, prescribed leg displacements, and combined strategies. The resulting mechanical systems exhibit periodic gaits whose stability characteristics are due to intermittent foot contact, and are largely determined by geometrical criteria. Most strikingly, we show that mechanics alone can confer asymptotic stability in heading and body orientation. In a companion paper, we apply our results to rapidly running cockroaches. Received: 6 September 1999 / Accepted in revised form: 8 May 2000     

Similar Crowd Behavior in Organisms of Vastly Different Body Size

Crowd behaviors can have large fitness consequences for social organisms. Here we ask if there are similarities in the crowd dynamics of organisms that differ in body size, manner of locomotion, cognitive abilities, and state of alarm. Existing models of human crowd behavior have not been tested for their generality across species and body size nor across routine and emergency movements. We explore this issue by comparing the traffic dynamics of humans and of Argentine ants (Linepithema humile) to the predictions of our own model which was designed to simulate pedestrian movement. Some parameter values in the model were directly measured on ants but others were allometrically scaled from the human values to ant values based on the body mass difference. The model, with appropriately scaled parameters, correctly predicted two important properties of crowd behaviour for both organisms in a variety of circumstances: the flow rates and the distribution of time headways between successive ants in the escape sequence. The ability of a model of human pedestrian dynamics to predict behaviours of ant aggregations through allometric scaling of some parameter values suggests that there are fundamental features of crowd behavior that transcend the biological idiosyncrasies of the organisms involved.     

Modeling observed chaotic oscillations in bursting neurons: the role of calcium dynamics and IP3

We study the dynamics and stability of legged locomotion in the horizontal plane. We discuss the relevance of idealized mechanical models, developed in a companion paper, to recent experiments and simulations on insect running and turning. Applying our results to rapidly running cockroaches, we show that the models' gait and force characteristics match observations reasonably well. Received: 6 September 1999 / Accepted in revised form: 8 May 2000     

Mechanical models for insect locomotion: active muscles and energy losses

 We extend the analysis of simple, energy-conserving models for the dynamics of insect locomotion in the horizontal plane developed in Schmitt and Holmes (2000a,b, 2001), where gaits characteristic of steady cockroach running and turning were evoked. In this paper, we include dissipation and energy inputs via active “muscles” in three forms: via prescribed torques at the “hip” pivot, via an active spring element of variable length, and via a pair of Hill-type muscle models representing an extensor/flexor system. Due to mechanical feedback of passive elastic forces, the stable gaits of the conservative models are preserved, and now energy input and absorption balances to additionally stabilize a preferred speed, with only modest neural sensing and feedback being required. However, these bipedal models still cannot simultaneously match observed moment-yaw magnitudes and fore-aft dynamics. Received: 17 September 2001 / Accepted: 20 February 2003 / Published online: 20 May 2003 Correspondence to: P. Holmes (e-mail: pholmes@math.Princeton.EDU) Acknowledgements. This work was supported by DARPA/ONR: N00014-98-1-0747 and DoE: DE-FG02-95ER25238. John Schmitt was partially supported by a DoD Graduate Fellowship, a Wu Fellowship of the School of Engineering and Applied Science, and a George Van Ness Lothrop Honorific Fellowship of the Graduate School at Princeton University. We thank Kenneth Meijer for allowing us to use his muscle model in Sect. 4 and Bob Full and Dan Koditschek for numerous helpful suggestions.     

Self-polarization and directional motility of cytoplasm

Background: Directional cell motility implies the presence of a steering mechanism and a functional asymmetry between the front and rear of the cell. How this functional asymmetry arises and is maintained during cell locomotion is, however, unclear. Lamellar fragments of fish epidermal keratocytes, which lack nuclei, microtubules and most organelles, present a simplified, perhaps minimal, system for analyzing this problem because they consist of little other than the motile machinery enclosed by a membrane and yet can move with remarkable speed and persistence.Results: We have produced two types of cellular fragments: discoid stationary fragments and polarized fragments undergoing locomotion. The organization and dynamics of the actin–myosin II system were isotropic in stationary fragments and anisotropic in the moving fragments. To investigate whether the creation of asymmetry could result in locomotion, a transient mechanical stimulus was applied to stationary fragments. The stimulus induced localized contraction and the formation of an actin–myosin II bundle at one edge of the fragment. Remarkably, stimulated fragments started to undergo locomotion and the locomotion and associated anisotropic organization of the actin–myosin II system were sustained after withdrawal of the stimulus.Conclusions: We propose a model in which lamellar cytoplasm is considered a dynamically bistable system capable of existing in a non-polarized or polarized state and interconvertible by mechanical stimulus. The model explains how the anisotropic organization of the lamellum is maintained in the process of locomotion. Polarized locomotion is sustained through a positive-feedback loop intrinsic to the actin–myosin II machinery: anisotropic organization of the machinery drives translocation, which then reinforces the asymmetry of the machinery, favoring further translocation.     

Evaluating functional roles of phase resetting in generation of adaptive human bipedal walking with a physiologically based model of the spinal pattern generator

The central pattern generators (CPGs) in the spinal cord strongly contribute to locomotor behavior. To achieve adaptive locomotion, locomotor rhythm generated by the CPGs is suggested to be functionally modulated by phase resetting based on sensory afferent or perturbations. Although phase resetting has been investigated during fictive locomotion in cats, its functional roles in actual locomotion have not been clarified. Recently, simulation studies have been conducted to examine the roles of phase resetting during human bipedal walking, assuming that locomotion is generated based on prescribed kinematics and feedback control. However, such kinematically based modeling cannot be used to fully elucidate the mechanisms of adaptation. In this article we proposed a more physiologically based mathematical model of the neural system for locomotion and investigated the functional roles of phase resetting. We constructed a locomotor CPG model based on a two-layered hierarchical network model of the rhythm generator (RG) and pattern formation (PF) networks. The RG model produces rhythm information using phase oscillators and regulates it by phase resetting based on foot-contact information. The PF model creates feedforward command signals based on rhythm information, which consists of the combination of five rectangular pulses based on previous analyses of muscle synergy. Simulation results showed that our model establishes adaptive walking against perturbing forces and variations in the environment, with phase resetting playing important roles in increasing the robustness of responses, suggesting that this mechanism of regulation may contribute to the generation of adaptive human bipedal locomotion.     

Learning the dynamics of reaching movements results in the modification of arm impedance and long-latency perturbation responses

 Some characteristics of arm movements that humans exhibit during learning the dynamics of reaching are consistent with a theoretical framework where training results in motor commands that are gradually modified to predict and compensate for novel forces that may act on the hand. As a first approximation, the motor control system behaves as an adapting controller that learns an internal model of the dynamics of the task. It approximates inverse dynamics and predicts motor commands that are appropriate for a desired limb trajectory. However, we had previously noted that subtle motion characteristics observed during changes in task dynamics challenged this simple model and raised the possibility that adaptation also involved sensory–motor feedback pathways. These pathways reacted to sensory feedback during the course of the movement. Here we hypothesize that adaptation to dynamics might also involve a modification of how the CNS responds to sensory feedback. We tested this through experiments that quantified how the motor system's response to errors during voluntary movements changed as it adapted to dynamics of a force field. We describe a nonlinear approach that approximates the impedance of the arm, i.e., force response as a function of arm displacement trajectory. We observe that after adaptation, the impedance function changes in a way that closely matches and counters the effect of the force field. This is particularly prominent in the long-latency (>100 ms) component of response to perturbations. Therefore, it appears that practice not only modifies the internal model with which the brain generates motor commands that initiate a movement, but also the internal model with which sensory feedback is integrated with the ongoing descending commands in order to respond to error during the movement. Received: 10 January 2001 / Accepted in revised form: 30 May 2001     

Spring-mass running: simple approximate solution and application to gait stability

The planar spring-mass model is frequently used to describe bouncing gaits (running, hopping, trotting, galloping) in animal and human locomotion and robotics. Although this model represents a rather simple mechanical system, an analytical solution predicting the center of mass trajectory during stance remains open. We derive an approximate solution in elementary functions assuming a small angular sweep and a small spring compression during stance. The predictive power and quality of this solution is investigated for model parameters relevant to human locomotion. The analysis shows that (i), for spring compressions of up to 20% (angle of attack > or = 60 degree, angular sweep < or = 60 degree) the approximate solution describes the stance dynamics of the center of mass within a 1% tolerance of spring compression and 0.6 degree tolerance of angular motion compared to numerical calculations, and (ii), despite its relative simplicity, the approximate solution accurately predicts stable locomotion well extending into the physiologically reasonable parameter domain. (iii) Furthermore, in a particular case, an explicit parametric dependency required for gait stability can be revealed extending an earlier, empirically found relationship. It is suggested that this approximation of the planar spring-mass dynamics may serve as an analytical tool for application in robotics and further research on legged locomotion.     

Neural control of Caenorhabditis elegans forward locomotion: the role of sensory feedback

This paper presents a simple yet biologically-grounded model for the neural control of Caenorhabditis elegans forward locomotion. We identify a minimal circuit within the C. elegans ventral cord that is likely to be sufficient to generate and sustain forward locomotion in vivo. This limited subcircuit appears to contain no obvious central pattern generated control. For that subcircuit, we present a model that relies on a chain of oscillators along the body which are driven by local and proximate mechano-sensory input. Computer simulations were used to study the model under a variety of conditions and to test whether it is behaviourally plausible. Within our model, we find that a minimal circuit of AVB interneurons and B-class motoneurons is sufficient to generate and sustain fictive forward locomotion patterns that are robust to significant environmental perturbations. The model predicts speed and amplitude modulation by the AVB command interneurons. An extended model including D-class motoneurons is included for comparison. Electronic supplementary material The online version of this article (doi:) contains supplementary material, which is available to authorized users. John Bryden and Netta Cohen contributed equally to this work.     

A mechanical link model of two-toed sloths: no pendular mechanics during suspensory locomotion

Sloths are morphologically specialized in suspensory quadrupedal locomotion and posture. During steady-state locomotion they utilize a trot-like footfall sequence. Contrasting the growing amount of published accounts of the functional morphology and kinematics of sloth locomotion, no study concerned with the dynamics of their quadrupedal suspensory locomotion has been conducted. Brachiating primates have been shown to travel at low mechanical costs using pendular mechanics, but this is associated with considerable dynamic forces exerted onto the support. To test whether sloth locomotion can be described by simple connected pendulum mechanics, we analyzed the dynamics of sloth locomotion with use of a mechanical segment link model. The model integrates the body segment parameters and is driven by kinematic data with both segment parameters and kinematic data obtained from the same sloth individual. No simple pendular mechanics were present. We then used the model to carry out an inverse dynamic analysis. The analysis allowed us to estimate net limb joint torques and substrate reaction forces during the contact phases. Predominant flexing limb joint torque profiles in the shoulder, elbow, hip, and knee are in stark contrast to published dominant extensor torques in the limb joints of pronograde quadrupedal mammals. This dissimilarity likely reflects the inverse orientation of the sloth towards the gravity vector. Nevertheless, scapular pivot and shoulder seem to provide the strongest torque for progression as expected based on unchanged basic kinematic pattern previously described. Our model predicts that sloths actively reduce the dynamical forces and moments that are transmitted onto the support. We conclude that these findings reflect the need to reduce the risk of breaking supports because in this case sloths would likely be unable to react quickly enough to prevent potentially lethal falls. To achieve this, sloths seem to avoid the dynamical consequences of effective pendular mechanics.     

Simple models for excitable and oscillatory neural networks

 Chains of coupled oscillators of simple “rotator” type have been used to model the central pattern generator (CPG) for locomotion in lamprey, among numerous applications in biology and elsewhere. In this paper, motivated by experiments on lamprey CPG with brainstem attached, we investigate a simple oscillator model with internal structure which captures both excitable and bursting dynamics. This model, and that for the coupling functions, is inspired by the Hodgkin–Huxley equations and two-variable simplifications thereof. We analyse pairs of coupled oscillators with both excitatory and inhibitory coupling. We also study traveling wave patterns arising from chains of oscillators, including simulations of “body shapes” generated by a double chain of oscillators providing input to a kinematic musculature model of lamprey.. Received: 25 November 1996 / Revised version: 9 December 1997     

White-noise stimulation of the Hodgkin–Huxley model

 We studied the combined influence of noise and constant current stimulations on the Hodgkin–Huxley neuron model through time and frequency analysis of the membrane-potential dynamics. We observed that, in agreement with experimental data (Guttman et al. 1974), at low noise and low constant current stimulation the behavior of the model is well approximated by that of the linearized Hodgkin–Huxley system. Conversely, nonlinearities due to firing dominate at large noise or current stimulations. The transition between the two regimes is abrupt, and takes place in the same range of noise and current intensities as the noise-induced transition characterized by the qualitative change in the stationary distribution of the membrane potential (Tanabe and Pakdaman 2001a). The implications of these results are discussed. Received: 27 July 2001 / Accepted in revised form: 18 December 2001     

Noise-induced transition in excitable neuron models

 We studied the influence of noisy stimulation on the Hodgkin-Huxley neuron model. Rather than examining the noise-related variability of the discharge times of the model – as has been done previously – our study focused on the effect of noise on the stationary distributions of the membrane potential and gating variables of the model. We observed that a gradual increase in the noise intensity did not result in a gradual change of the distributions. Instead, we could identify a critical intermediate noise range in which the shapes of the distributions underwent a drastic qualitative change. Namely, they moved from narrow unimodal Gaussian-like shapes associated with low noise intensities to ones that spread widely at large noise intensities. In particular, for the membrane potential and the sodium activation variable, the distributions changed from unimodal to bimodal. Thus, our investigation revealed a noise-induced transition in the Hodgkin-Huxley model. In order to further characterize this phenomenon, we considered a reduced one-dimensional model of an excitable system, namely the active rotator. For this model, our analysis indicated that the noise-induced transition is associated with a deterministic bifurcation of approximate equations governing the dynamics of the mean and variance of the state variable. Finally, we shed light on the possible functional importance of this noise-induced transition in neuronal coding by determining its effect on the spike timing precision in models of neuronal ensembles. Received: 19 September 2000 / Accepted in revised form: 4 March 2001     

Bipedal walking and running with spring-like biarticular muscles

Compliant elements in the leg musculoskeletal system appear to be important not only for running but also for walking in human locomotion as shown in the energetics and kinematics studies of spring-mass model. While the spring-mass model assumes a whole leg as a linear spring, it is still not clear how the compliant elements of muscle-tendon systems behave in a human-like segmented leg structure. This study presents a minimalistic model of compliant leg structure that exploits dynamics of biarticular tension springs. In the proposed bipedal model, each leg consists of three leg segments with passive knee and ankle joints that are constrained by four linear tension springs. We found that biarticular arrangements of the springs that correspond to rectus femoris, biceps femoris and gastrocnemius in human legs provide self-stabilizing characteristics for both walking and running gaits. Through the experiments in simulation and a real-world robotic platform, we show how behavioral characteristics of the proposed model agree with basic patterns of human locomotion including joint kinematics and ground reaction force, which could not be explained in the previous models.     

Conservation rules, their breakdown, and optimality in Caenorhabditis sinusoidal locomotion

Undulatory locomotion is common to nematodes as well as to limbless vertebrates, but its control is not understood in spite of the identification of hundred of genes involved in Caenorhabditis elegans locomotion. To reveal the mechanisms of nematode undulatory locomotion, we quantitatively analysed the movement of C. elegans with genetic perturbations to neurons, muscles, and skeleton (cuticle). We also compared locomotion of different Caenorhabditis species. We constructed a theoretical model that combines mechanics and biophysics, and that is constrained by the observations of propulsion and muscular velocities, as well as wavelength and amplitude of undulations. We find that normalized wavelength is a conserved quantity among wild-type C. elegans individuals, across mutants, and across different species. The velocity of forward propulsion scales linearly with the velocity of the muscular wave and the corresponding slope is also a conserved quantity and almost optimal; the exceptions are in some mutants affecting cuticle structure. In theoretical terms, the optimality of the slope is equivalent to the exact balance between muscular and visco-elastic body reaction bending moments. We find that the amplitude and frequency of undulations are inversely correlated and provide a theoretical explanation for this fact. These experimental results are valid both for young adults and for all larval stages of wild-type C. elegans. In particular, during development, the amplitude scales linearly with the wavelength, consistent with our theory. We also investigated the influence of substrate firmness on motion parameters, and found that it does not affect the above invariants. In general, our biomechanical model can explain the observed robustness of the mechanisms controlling nematode undulatory locomotion.     

Mathematical model for the effects of adhesion and mechanics on cell migration speed.

Migration of mammalian blood and tissue cells over adhesive surfaces is apparently mediated by specific reversible reactions between cell membrane adhesion receptors and complementary ligands attached to the substratum. Although in a number of systems these receptors and ligand molecules have been isolated and identified, a theory capable of predicting the effects of their properties on cell migration behavior currently does not exist. We present a simple mathematical model for elucidating the dependence of cell speed on adhesion-receptor/ligand binding and cell mechanical properties. Our model can be applied to propose answers to questions such as: does an optimal adhesiveness exist for cell movement? How might changes in receptor and ligand density and/or affinity affect the rate of migration? Can cell rheological properties influence movement speed? This model incorporates cytoskeletal force generation, cell polarization, and dynamic adhesion as requirements for persistent cell movement. A critical feature is the proposed existence of an asymmetry in some cell adhesion-receptor property, correlated with cell polarity. We consider two major alternative mechanisms underlying this asymmetry: (a) a spatial distribution of adhesion-receptor number due to polarized endocytic trafficking and (b) a spatial variation in adhesion-receptor/ligand bond strength. Applying a viscoelastic-solid model for cell mechanics allows us to represent one-dimensional locomotion with a system of differential equations describing cell deformation and displacement along with adhesion-receptor dynamics. In this paper, we solve these equations under the simplifying assumption that receptor dynamics are at a quasi-steady state relative to cell locomotion. Thus, our results are strictly valid for sufficiently slow cell movement, as typically observed for tissue cells such as fibroblasts. Numerical examples relevant to experimental systems are provided. Our results predict how cell speed might vary with intracellular contractile force, cell rheology, receptor/ligand kinetics, and receptor/ligand number densities. A biphasic dependence is shown to be possible with respect to some of the system parameters, with position of the maxima essentially governed by a balance between transmitted contractile force and adhesiveness. We demonstrate that predictions for the two alternative asymmetry mechanisms can be distinguished and could be experimentally tested using cell populations possessing different adhesion-receptor numbers.     

Density-dependent birth rate, birth pulses and their population dynamic consequences

 In most models of population dynamics, increases in population due to birth are assumed to be time-independent, but many species reproduce only during a single period of the year. We propose a single-species model with stage structure for the dynamics in a wild animal population for which births occur in a single pulse once per time period. Using the discrete dynamical system determined by the stroboscopic map, we obtain an exact periodic solution of systems which are with Ricker functions or Beverton-Holt functions, and obtain the threshold conditions for their stability. Above this threshold, there is a characteristic sequence of bifurcations, leading to chaotic dynamics, which implies that the dynamical behaviors of the single species model with birth pulses are very complex, including small-amplitude annual oscillations, large-amplitude multi-annual cycles, and chaos. This suggests that birth pulse, in effect, provides a natural period or cyclicity that allows for a period-doubling route to chaos. Received: 13 June 2001 / Revised version: 7 September 2001 / Published online: 8 February 2002