The role of feedback in morphological computation with compliant bodies

The generation of robust periodic movements of complex nonlinear robotic systems is inherently difficult, especially, if parts of the robots are compliant. It has previously been proposed that complex nonlinear features of a robot, similarly as in biological organisms, might possibly facilitate its control. This bold hypothesis, commonly referred to as morphological computation, has recently received some theoretical support by Hauser et?al. (Biol Cybern 105:355–370, doi:10.1007/s00422-012-0471-0, 2012). We show in this article that this theoretical support can be extended to cover not only the case of fading memory responses to external signals, but also the essential case of autonomous generation of adaptive periodic patterns, as, e.g., needed for locomotion. The theory predicts that feedback into the morphological computing system is necessary and sufficient for such tasks, for which a fading memory is insufficient. We demonstrate the viability of this theoretical analysis through computer simulations of complex nonlinear mass–spring systems that are trained to generate a large diversity of periodic movements by adapting the weights of a simple linear feedback device. Hence, the results of this article substantially enlarge the theoretically tractable application domain of morphological computation in robotics, and also provide new paradigms for understanding control principles of biological organisms.     

Automated Design of Complex Dynamic Systems

Several fields of study are concerned with uniting the concept of computation with that of the design of physical systems. For example, a recent trend in robotics is to design robots in such a way that they require a minimal control effort. Another example is found in the domain of photonics, where recent efforts try to benefit directly from the complex nonlinear dynamics to achieve more efficient signal processing. The underlying goal of these and similar research efforts is to internalize a large part of the necessary computations within the physical system itself by exploiting its inherent non-linear dynamics. This, however, often requires the optimization of large numbers of system parameters, related to both the system''s structure as well as its material properties. In addition, many of these parameters are subject to fabrication variability or to variations through time. In this paper we apply a machine learning algorithm to optimize physical dynamic systems. We show that such algorithms, which are normally applied on abstract computational entities, can be extended to the field of differential equations and used to optimize an associated set of parameters which determine their behavior. We show that machine learning training methodologies are highly useful in designing robust systems, and we provide a set of both simple and complex examples using models of physical dynamical systems. Interestingly, the derived optimization method is intimately related to direct collocation a method known in the field of optimal control. Our work suggests that the application domains of both machine learning and optimal control have a largely unexplored overlapping area which envelopes a novel design methodology of smart and highly complex physical systems.     

Computational aspects of motion perception in natural and artificial vision systems.

In this paper a computational scheme for motion perception in artificial and natural vision systems is described. The scheme is motivated by a mathematical analysis in which first-order spatial properties of optical flow, such as singular points and elementary components of optical flow, are shown to be salient features for the computation and analysis of visual motion. The fact that different methods for the computation of optical flow produce similar results is explained in terms of the simple spatial structure of the image motion of rigid bodies. Singular points and elementary flow components are used to compute motion parameters, such as time-to-collision and angular velocity, and also to segment the visual field into areas which correspond to different motions. Then a number of biological implications are discussed. Electrophysiological findings suggest that the brain perceives visual motion by detecting and analysing optical flow components. However, the cortical neurons, which seem to detect elementary flow components, are not able to extract these components from more complex flows. A simple model for the organization of the receptive field of these cells, which is consistent with anatomical and electrophysiological data, is described at the end of the paper.     

Understanding complex systems: lessons from Auzoux’s and von Hagens’s anatomical models

Animal and human anatomy is among the most complex systems known, and suitable teaching methods have been of great importance in the progress of knowledge. Examining the human body is part of the process by which medical students come to understand living forms. However, the need to preserve cadavers has led to the development of various techniques to manufacture models for teaching purposes. A variety of materials, such as wax, wood, papier-maché, or glass, have long been used to construct animal and plant models. In the case of the human body, the most innovative, yet controversial, method of preservation has been plastination, invented by the German physician Gunther von Hagens, by which actual human bodies are preserved as odourless and aesthetic models for teaching and exhibitions. We point out in our study that the ‘hands-on’ approach that some anatomical models allow, namely, the (clastic) disassembly and reassembly of the parts of complex systems and their models, is not only a crucial tool for learning, but is far superior to the simple passive observation that rigid, single-piece models allow. And what is valid for the learning of anatomy can be generalized to the acquisition of knowledge of other complex physical systems.     

Human attention: the exclusion of distracting information as a function of real and apparent separation of relevant and irrelevant events

A leading theory suggests that human vision operates by separate and parallel analysis of each of the simple features of the visual scene; but computation of the identity of objects, which needs combination of these features, is undertaken selectively and only for some parts of the incoming information. It is known that the identity of distracting events affects reaction time only for distractors spatially close to the reaction signal. This suggests that the selection is spatially based; perhaps taking place in the projection areas. Experiments on this topic have, however, normally considered events only at a fixed viewing distance; in this study different viewing distances were employed. It was found that, over the range of conditions used, the ability to exclude irrelevant distractors depended upon the true physical separation and not on the angular separation at the eye of the observer. Hence the 'attention' system must operate at a point later than the computation of true physical location, rather than at the very early stage where angular separation only is available.     

Rethinking Causation for Data-intensive Biology: Constraints,Cancellations, and Quantized Organisms

Complex organisms thwart the simple rectilinear causality paradigm of “necessary and sufficient,” with its experimental strategy of “knock down and overexpress.” This Essay organizes the eccentricities of biology into four categories that call for new mathematical approaches; recaps for the biologist the philosopher's recent refinements to the causation concept and the mathematician's computational tools that handle some but not all of the biological eccentricities; and describes overlooked insights that make causal properties of physical hierarchies such as emergence and downward causation straightforward. Reviewing and extrapolating from similar situations in physics, it is suggested that new mathematical tools for causation analysis incorporating feedback, signal cancellation, nonlinear dependencies, physical hierarchies, and fixed constraints rather than instigative changes will reveal unconventional biological behaviors. These include “eigenisms,” organisms that are limited to quantized states; trajectories that steer a system such as an evolving species toward optimal states; and medical control via distributed “sheets” rather than single control points.     

Compartmental neural simulations with spatial adaptivity

Since their inception, computational models have become increasingly complex and useful counterparts to laboratory experiments within the field of neuroscience. Today several software programs exist to solve the underlying mathematical system of equations, but such programs typically solve these equations in all parts of a cell (or network of cells) simultaneously, regardless of whether or not all of the cell is active. This approach can be inefficient if only part of the cell is active and many simulations must be performed. We have previously developed a numerical method that provides a framework for spatial adaptivity by making the computations local to individual branches rather than entire cells (Rempe and Chopp, SIAM Journal on Scientific Computing, 28: 2139–2161, 2006). Once the computation is reduced to the level of branches instead of cells, spatial adaptivity is straightforward: the active regions of the cell are detected and computational effort is focused there, while saving computations in other regions of the cell that are at or near rest. Here we apply the adaptive method to four realistic neuronal simulation scenarios and demonstrate its improved efficiency over non-adaptive methods. We find that the computational cost of the method scales with the amount of activity present in the simulation, rather than the physical size of the system being simulated. For certain problems spatial adaptivity reduces the computation time by up to 80%.     

A mathematical model of the adaptive control of human arm motions

This paper discusses similarities between models of adaptive motor control suggested by recent experiments with human and animal subjects, and the structure of a new control law derived mathematically from nonlinear stability theory. In both models, the control actions required to track a specified trajectory are adaptively assembled from a large collection of simple computational elements. By adaptively recombining these elements, the controllers develop complex internal models which are used to compensate for the effects of externally imposed forces or changes in the physical properties of the system. On a motor learning task involving planar, multi-joint arm motions, the simulated performance of the mathematical model is shown to be qualitatively similar to observed human performance, suggesting that the model captures some of the interesting features of the dynamics of low-level motor adaptation. Received: 20 September 1994 / Accepted in revised form: 18 November 1998     

Fading memory and kernel properties of generic cortical microcircuit models

It is quite difficult to construct circuits of spiking neurons that can carry out complex computational tasks. On the other hand even randomly connected circuits of spiking neurons can in principle be used for complex computational tasks such as time-warp invariant speech recognition. This is possible because such circuits have an inherent tendency to integrate incoming information in such a way that simple linear readouts can be trained to transform the current circuit activity into the target output for a very large number of computational tasks. Consequently we propose to analyze circuits of spiking neurons in terms of their roles as analog fading memory and non-linear kernels, rather than as implementations of specific computational operations and algorithms. This article is a sequel to [W. Maass, T. Natschl?ger, H. Markram, Real-time computing without stable states: a new framework for neural computation based on perturbations, Neural Comput. 14 (11) (2002) 2531-2560, Online available as #130 from: ], and contains new results about the performance of generic neural microcircuit models for the recognition of speech that is subject to linear and non-linear time-warps, as well as for computations on time-varying firing rates. These computations rely, apart from general properties of generic neural microcircuit models, just on capabilities of simple linear readouts trained by linear regression. This article also provides detailed data on the fading memory property of generic neural microcircuit models, and a quick review of other new results on the computational power of such circuits of spiking neurons.     

Asymmetric and adaptive reward coding via normalized reinforcement learning

Learning is widely modeled in psychology, neuroscience, and computer science by prediction error-guided reinforcement learning (RL) algorithms. While standard RL assumes linear reward functions, reward-related neural activity is a saturating, nonlinear function of reward; however, the computational and behavioral implications of nonlinear RL are unknown. Here, we show that nonlinear RL incorporating the canonical divisive normalization computation introduces an intrinsic and tunable asymmetry in prediction error coding. At the behavioral level, this asymmetry explains empirical variability in risk preferences typically attributed to asymmetric learning rates. At the neural level, diversity in asymmetries provides a computational mechanism for recently proposed theories of distributional RL, allowing the brain to learn the full probability distribution of future rewards. This behavioral and computational flexibility argues for an incorporation of biologically valid value functions in computational models of learning and decision-making.     

Growth process and interfacial process considered in connection with specific surface area

Analytical expressions for the radius of gyration and maximum dimension of compound bodies formed from simple geometric elements are derived. Using principles of symmetry and of commonly observed features of globular protein structures, it is shown how a priori reasonable tertiary and quaternary structures can be distinguished. The formulae, which are simple to apply, permit a rapid comparison of relatively complex shapes, so reducing the amount of computation necessary for a detailed analysis of small angle scattering data. In conjunction with other biophysical data, the morphological parameters derived from scattering curves can be used to generate quite complex biologically reasonable models of protein structures.     

Convergent evolution and locomotion through complex terrain by insects, vertebrates and robots

Arthropods are the most successful members of the animal kingdom largely because of their ability to move efficiently through a range of environments. Their agility has not been lost on engineers seeking to design agile legged robots. However, one cannot simply copy mechanical and neural control systems from insects into robotic designs. Rather one has to select the properties that are critical for specific behaviors that the engineer wants to capture in a particular robot. Convergent evolution provides an important clue to the properties of legged locomotion that are critical for success. Arthropods and vertebrates evolved legged locomotion independently. Nevertheless, many neural control properties and mechanical schemes are remarkably similar. Here we describe three aspects of legged locomotion that are found in both insects and vertebrates and that provide enhancements to legged robots. They are leg specialization, body flexion and the development of a complex head structure. Although these properties are commonly seen in legged animals, most robotic vehicles have similar legs throughout, rigid bodies and rudimentary sensors on what would be considered the head region. We describe these convergent properties in the context of robots that we developed to capture the agility of insects in moving through complex terrain.     

Multisensory convergence in the mushroom bodies of ants and bees

The mushroom bodies, central neuropils in the arthropod brain, are involved in learning and memory and in the control of complex behavior. In most insects, the mushroom bodies receive direct olfactory input in their calyx region. In Hymenoptera, olfactory input is layered in the calyx. In ants, several layers can be discriminated that correspond to different clusters of glomeruli in the antennal lobes, perhaps corresponding to different classes of odors. Only in Hymenoptera, the mushroom body calyx also receives direct visual input from the optic lobes. In bees, six calycal layers receive input from different classes of visual interneurons, probably representing different parts of the visual field and different visual properties. Taken together, the mushroom bodies receive distinct multisensory information in many segregated input layers.     

Generalization in Adaptation to Stable and Unstable Dynamics

Humans skillfully manipulate objects and tools despite the inherent instability. In order to succeed at these tasks, the sensorimotor control system must build an internal representation of both the force and mechanical impedance. As it is not practical to either learn or store motor commands for every possible future action, the sensorimotor control system generalizes a control strategy for a range of movements based on learning performed over a set of movements. Here, we introduce a computational model for this learning and generalization, which specifies how to learn feedforward muscle activity in a function of the state space. Specifically, by incorporating co-activation as a function of error into the feedback command, we are able to derive an algorithm from a gradient descent minimization of motion error and effort, subject to maintaining a stability margin. This algorithm can be used to learn to coordinate any of a variety of motor primitives such as force fields, muscle synergies, physical models or artificial neural networks. This model for human learning and generalization is able to adapt to both stable and unstable dynamics, and provides a controller for generating efficient adaptive motor behavior in robots. Simulation results exhibit predictions consistent with all experiments on learning of novel dynamics requiring adaptation of force and impedance, and enable us to re-examine some of the previous interpretations of experiments on generalization.     

Swarm cognition on off-road autonomous robots

This paper contributes with the first validation of swarm cognition as a useful framework for the design of autonomous robots controllers. The proposed model is built upon the authors’ previous work validated on a simulated robot performing local navigation on a 2-D deterministic world. Based on the ant foraging metaphor and motivated by the multiple covert attention hypothesis, the model consists of a set of simple virtual agents inhabiting the robot’s visual input, searching in a collectively coordinated way for obstacles. Parsimonious and accurate visual attention, operating on a by-need basis, is attained by making the activity of these agents modulated by the robot’s action selection process. A by-product of the system is the maintenance of active, parallel and sparse spatial working memories. In short, the model exhibits the self-organisation of a relevant set of features composing a cognitive system. To show its robustness, the model is extended in this paper to handle the challenges of physical off-road robots equipped with noisy stereoscopic vision sensors. Furthermore, an extensive aggregate of biological arguments sustaining the model is provided. Experimental results show the ability of the model to robustly control the robot on a local navigation task, with less than 1% of the robot’s visual input being analysed. Hence, with this system the computational cost of perception is considerably reduced, thus fostering robot miniaturisation and energetic efficiency. This confirms the advantages of using a swarm-based system, operating in an intricate way with action selection, to judiciously control visual attention and maintain sparse spatial memories, constituting a basic form of swarm cognition.     

Fully decentralized control of a soft-bodied robot inspired by true slime mold

Animals exhibit astoundingly adaptive and supple locomotion under real world constraints. In order to endow robots with similar capabilities, we must implement many degrees of freedom, equivalent to animals, into the robots’ bodies. For taming many degrees of freedom, the concept of autonomous decentralized control plays a pivotal role. However a systematic way of designing such autonomous decentralized control system is still missing. Aiming at understanding the principles that underlie animals’ locomotion, we have focused on a true slime mold, a primitive living organism, and extracted a design scheme for autonomous decentralized control system. In order to validate this design scheme, this article presents a soft-bodied amoeboid robot inspired by the true slime mold. Significant features of this robot are twofold: (1) the robot has a truly soft and deformable body stemming from real-time tunable springs and protoplasm, the former is used for an outer skin of the body and the latter is to satisfy the law of conservation of mass; and (2) fully decentralized control using coupled oscillators with completely local sensory feedback mechanism is realized by exploiting the long-distance physical interaction between the body parts stemming from the law of conservation of protoplasmic mass. Simulation results show that this robot exhibits highly supple and adaptive locomotion without relying on any hierarchical structure. The results obtained are expected to shed new light on design methodology for autonomous decentralized control system.     

Neural network control of simple limb movements

It is possible to embed the control and computation of a simple single-joint movement at different speeds by a small non-linear network of neuron-like elements. The network "learns" by appropriate adjustment of the strengths of interconnection, or synaptic weights, between the neuron-like elements. The learning of a few movement trajectories is generalized to the learning of a family of unlearned trajectories. These observations are in support of our hypothesis that relaxation of a network from an initial state to a final equilibrium state is both causal and computational to movement generation and control.     

From embodied mind to embodied robotics: Humanities and system theoretical aspects

After an introduction (1) the article analyzes the evolution of the embodied mind (2), the innovation of embodied robotics (3), and finally discusses conclusions of embodied robotics for human responsibility (4). Considering the evolution of the embodied mind (2), we start with an introduction of complex systems and nonlinear dynamics (2.1), apply this approach to neural self-organization (2.2), distinguish degrees of complexity of the brain (2.3), explain the emergence of cognitive states by complex systems dynamics (2.4), and discuss criteria for modeling the brain as complex nonlinear system (2.5). The innovation of embodied robotics (3) is a challenge of future technology. We start with the distinction of symbolic and embodied AI (3.1) and explain embodied robots as dynamical systems (3.2). Self-organization needs self-control of technical systems (3.3). Cellular neural networks (CNN) are an example of self-organizing technical systems offering new avenues for neurobionics (3.4). In general, technical neural networks support different kinds of learning robots (3.5). Finally, embodied robotics aim at the development of cognitive and conscious robots (3.6).     

Cooperative object transport with a swarm of e-puck robots: robustness and scalability of evolved collective strategies

Cooperative object transport in distributed multi-robot systems requires the coordination and synchronisation of pushing/pulling forces by a group of autonomous robots in order to transport items that cannot be transported by a single agent. The results of this study show that fairly robust and scalable collective transport strategies can be generated by robots equipped with a relatively simple sensory apparatus (i.e. no force sensors and no devices for direct communication). In the experiments described in this paper, homogeneous groups of physical e-puck robots are required to coordinate and synchronise their actions in order to transport a heavy rectangular cuboid object as far as possible from its starting position to an arbitrary direction. The robots are controlled by dynamic neural networks synthesised using evolutionary computation techniques. The best evolved controller demonstrates an effective group transport strategy that is robust to variability in the physical characteristics of the object (i.e. object mass and size of the longest object’s side) and scalable to different group sizes. To run these experiments, we designed, built, and mounted on the robots a new sensor that returns the agents’ displacement on a 2D plane. The study shows that the feedback generated by the robots’ sensors relative to the object’s movement is sufficient to allow the robots to coordinate their efforts and to sustain the transports for an extended period of time. By extensively analysing successful behavioural strategies, we illustrate the nature of the operational mechanisms underpinning the coordination and synchronisation of actions during group transport.     

Modeling locomotion of a soft-bodied arthropod using inverse dynamics

Most bio-inspired robots have been based on animals with jointed, stiff skeletons. There is now an increasing interest in mimicking the robust performance of animals in natural environments by incorporating compliant materials into the locomotory system. However, the mechanics of moving, highly conformable structures are particularly difficult to predict. This paper proposes a planar, extensible-link model for the soft-bodied tobacco hornworm caterpillar, Manduca sexta, to provide insight for biologists and engineers studying locomotion by highly deformable animals and caterpillar-like robots. Using inverse dynamics to process experimentally acquired point-tracking data, ground reaction forces and internal forces were determined for a crawling caterpillar. Computed ground reaction forces were compared to experimental data to validate the model. The results show that a system of linked extendable joints can faithfully describe the general form and magnitude of the contact forces produced by a crawling caterpillar. Furthermore, the model can be used to compute internal forces that cannot be measured experimentally. It is predicted that between different body segments in stance phase the body is mostly kept in tension and that compression only occurs during the swing phase when the prolegs release their grip. This finding supports a recently proposed mechanism for locomotion by soft animals in which the substrate transfers compressive forces from one part of the body to another (the environmental skeleton) thereby minimizing the need for hydrostatic stiffening. The model also provides a new means to characterize and test control strategies used in caterpillar crawling and soft robot locomotion.