Information-driven self-organization: the dynamical system approach to autonomous robot behavior

In recent years, information theory has come into the focus of researchers interested in the sensorimotor dynamics of both robots and living beings. One root for these approaches is the idea that living beings are information processing systems and that the optimization of these processes should be an evolutionary advantage. Apart from these more fundamental questions, there is much interest recently in the question how a robot can be equipped with an internal drive for innovation or curiosity that may serve as a drive for an open-ended, self-determined development of the robot. The success of these approaches depends essentially on the choice of a convenient measure for the information. This article studies in some detail the use of the predictive information (PI), also called excess entropy or effective measure complexity, of the sensorimotor process. The PI of a process quantifies the total information of past experience that can be used for predicting future events. However, the application of information theoretic measures in robotics mostly is restricted to the case of a finite, discrete state-action space. This article aims at applying the PI in the dynamical systems approach to robot control. We study linear systems as a first step and derive exact results for the PI together with explicit learning rules for the parameters of the controller. Interestingly, these learning rules are of Hebbian nature and local in the sense that the synaptic update is given by the product of activities available directly at the pertinent synaptic ports. The general findings are exemplified by a number of case studies. In particular, in a two-dimensional system, designed at mimicking embodied systems with latent oscillatory locomotion patterns, it is shown that maximizing the PI means to recognize and amplify the latent modes of the robotic system. This and many other examples show that the learning rules derived from the maximum PI principle are a versatile tool for the self-organization of behavior in complex robotic systems.     

Insect-inspired high-speed motion vision system for robot control

The mechanism for motion detection in a fly’s vision system, known as the Reichardt correlator, suffers from a main shortcoming as a velocity estimator: low accuracy. To enable accurate velocity estimation, responses of the Reichardt correlator to image sequences are analyzed in this paper. An elaborated model with additional preprocessing modules is proposed. The relative error of velocity estimation is significantly reduced by establishing a real-time response-velocity lookup table based on the power spectrum analysis of the input signal. By exploiting the improved velocity estimation accuracy and the simple structure of the Reichardt correlator, a high-speed vision system of 1?kHz is designed and applied for robot yaw-angle control in real-time experiments. The experimental results demonstrate the potential and feasibility of applying insect-inspired motion detection to robot control.     

A real-time spiking cerebellum model for learning robot control

We describe a neural network model of the cerebellum based on integrate-and-fire spiking neurons with conductance-based synapses. The neuron characteristics are derived from our earlier detailed models of the different cerebellar neurons. We tested the cerebellum model in a real-time control application with a robotic platform. Delays were introduced in the different sensorimotor pathways according to the biological system. The main plasticity in the cerebellar model is a spike-timing dependent plasticity (STDP) at the parallel fiber to Purkinje cell connections. This STDP is driven by the inferior olive (IO) activity, which encodes an error signal using a novel probabilistic low frequency model. We demonstrate the cerebellar model in a robot control system using a target-reaching task. We test whether the system learns to reach different target positions in a non-destructive way, therefore abstracting a general dynamics model. To test the system's ability to self-adapt to different dynamical situations, we present results obtained after changing the dynamics of the robotic platform significantly (its friction and load). The experimental results show that the cerebellar-based system is able to adapt dynamically to different contexts.     

Proline and glycine control protein self-organization into elastomeric or amyloid fibrils

Elastin provides extensible tissues, including arteries and skin, with the propensity for elastic recoil, whereas amyloid fibrils are associated with tissue-degenerative diseases, such as Alzheimer's. Although both elastin-like and amyloid-like materials result from the self-organization of proteins into fibrils, the molecular basis of their differing physical properties is poorly understood. Using molecular simulations of monomeric and aggregated states, we demonstrate that elastin-like and amyloid-like peptides are separable on the basis of backbone hydration and peptide-peptide hydrogen bonding. The analysis of diverse sequences, including those of elastin, amyloids, spider silks, wheat gluten, and insect resilin, reveals a threshold in proline and glycine composition above which amyloid formation is impeded and elastomeric properties become apparent. The predictive capacity of this threshold is confirmed by the self-assembly of recombinant peptides into either amyloid or elastin-like fibrils. Our findings support a unified model of protein aggregation in which hydration and conformational disorder are fundamental requirements for elastomeric function.     

Humanoid robot Lola: Design and walking control

In this paper we present the humanoid robot LOLA, its mechatronic hardware design, simulation and real-time walking control. The goal of the LOLA-project is to build a machine capable of stable, autonomous, fast and human-like walking. LOLA is characterized by a redundant kinematic configuration with 7-DoF legs, an extremely lightweight design, joint actuators with brushless motors and an electronics architecture using decentralized joint control. Special emphasis was put on an improved mass distribution of the legs to achieve good dynamic performance. Trajectory generation and control aim at faster, more flexible and robust walking. Center of mass trajectories are calculated in real-time from footstep locations using quadratic programming and spline collocation methods. Stabilizing control uses hybrid position/force control in task space with an inner joint position control loop. Inertial stabilization is achieved by modifying the contact force trajectories.     

Mathematical model for the self-organization of neural networks

Mutual inhibition between neurons combined with a learning principle similar to that proposed by Hebb is shown to secure a powerful selforganizing property for neural networks. Numerical analysis reveals that the system investigated always organizes itself into the same final state from any arbitrarily chosen initial state.     

Comparison of human and humanoid robot control of upright stance

There is considerable recent interest in developing humanoid robots. An important substrate for many motor actions in both humans and biped robots is the ability to maintain a statically or dynamically stable posture. Given the success of the human design, one would expect there are lessons to be learned in formulating a postural control mechanism for robots. In this study we limit ourselves to considering the problem of maintaining upright stance. Human stance control is compared to a suggested method for robot stance control called zero moment point (ZMP) compensation. Results from experimental and modeling studies suggest there are two important subsystems that account for the low- and mid-frequency (DC to 1 Hz) dynamic characteristics of human stance control. These subsystems are (1) a “sensory integration” mechanism whereby orientation information from multiple sensory systems encoding body kinematics (i.e. position, velocity) is flexibly combined to provide an overall estimate of body orientation while allowing adjustments (sensory re-weighting) that compensate for changing environmental conditions and (2) an “effort control” mechanism that uses kinetic-related (i.e., force-related) sensory information to reduce the mean deviation of body orientation from upright. Functionally, ZMP compensation is directly analogous to how humans appear to use kinetic feedback to modify the main sensory integration feedback loop controlling body orientation. However, a flexible sensory integration mechanism is missing from robot control leaving the robot vulnerable to instability in conditions where humans are able to maintain stance. We suggest the addition of a simple form of sensory integration to improve robot stance control. We also investigate how the biological constraint of feedback time delay influences the human stance control design. The human system may serve as a guide for improved robot control, but should not be directly copied because the constraints on robot and human control are different.     

Common Variants for Heart Failure

Heart failure (HF) is a common disease with high morbidity and mortality; however, none of thedrugs available are now entirely optimal for the treatment of HF. In addition to various clinical diseases andenvironment influences, genetic factors also contribute to the development and progression of HF. Identifyingthe common variants for HF by genome-wide association studies will facilitate the understanding ofpathophysiological mechanisms underlying HF. This review summarizes the recently identified commonvariants for HF risk and outcome and discusses their implications for the clinic therapy.     

The self-organization of speech sounds

The speech code is a vehicle of language: it defines a set of forms used by a community to carry information. Such a code is necessary to support the linguistic interactions that allow humans to communicate. How then may a speech code be formed prior to the existence of linguistic interactions? Moreover, the human speech code is discrete and compositional, shared by all the individuals of a community but different across communities, and phoneme inventories are characterized by statistical regularities. How can a speech code with these properties form? We try to approach these questions in the paper, using the "methodology of the artificial". We build a society of artificial agents, and detail a mechanism that shows the formation of a discrete speech code without pre-supposing the existence of linguistic capacities or of coordinated interactions. The mechanism is based on a low-level model of sensory-motor interactions. We show that the integration of certain very simple and non-language-specific neural devices leads to the formation of a speech code that has properties similar to the human speech code. This result relies on the self-organizing properties of a generic coupling between perception and production within agents, and on the interactions between agents. The artificial system helps us to develop better intuitions on how speech might have appeared, by showing how self-organization might have helped natural selection to find speech.     

A model of myoglobin self-organization

The self-organization of helical regions of myoglobin into a compact tertiary structure is considered on the basis of the hypothesis on the step-wise mechanism of self-organization of protein molecules. It is assumed that the self-organization begins with the formation of “ centers of crystallization ” and proceeds with the growth of one such center or by a sequential collapse of two or more grown centers.Different pathways of self-organization of myoglobin are considered; the most favourable structures corresponding to the greatest number of dehydrated bulky hydroptiobic groups and to all the strongly hydrophilic groups exposed to water are selected at every stage of the given pathway and the others are neglected. One of the two most favourable structures obtained in such a way coincides in rough resolution with the native tertiary structure of protein.     

Full-state tracking control of a mobile robot using neural networks

In this paper a nonholonomic mobile robot with completely unknown dynamics is discussed. A mathematical model has been considered and an efficient neural network is developed, which ensures guaranteed tracking performance leading to stability of the system. The neural network assumes a single layer structure, by taking advantage of the robot regressor dynamics that expresses the highly nonlinear robot dynamics in a linear form in terms of the known and unknown robot dynamic parameters. No assumptions relating to the boundedness is placed on the unmodeled disturbances. It is capable of generating real-time smooth and continuous velocity control signals that drive the mobile robot to follow the desired trajectories. The proposed approach resolves speed jump problem existing in some previous tracking controllers. Further, this neural network does not require offline training procedures. Lyapunov theory has been used to prove system stability. The practicality and effectiveness of the proposed tracking controller are demonstrated by simulation and comparison results.     

AutoMoDe: A novel approach to the automatic design of control software for robot swarms

We introduce AutoMoDe: a novel approach to the automatic design of control software for robot swarms. The core idea in AutoMoDe recalls the approach commonly adopted in machine learning for dealing with the bias–variance tradedoff: to obtain suitably general solutions with low variance, an appropriate design bias is injected. AutoMoDe produces robot control software by selecting, instantiating, and combining preexisting parametric modules—the injected bias. The resulting control software is a probabilistic finite state machine in which the topology, the transition rules and the values of the parameters are obtained automatically via an optimization process that maximizes a task-specific objective function. As a proof of concept, we define AutoMoDe-Vanilla, which is a specialization of AutoMoDe for the e-puck robot. We use AutoMoDe-Vanilla to design the robot control software for two different tasks: aggregation and foraging. The results show that the control software produced by AutoMoDe-Vanilla (i) yields good results, (ii) appears to be robust to the so called reality gap, and (iii) is naturally human-readable.     

Changes in stimulus control during guided skill learning in rats

We examined the changes in stimulus control occurring during guided skill learning in rats. Twenty rats were trained to complete a left-right sequence of lever presses guided by the onset and offset of panel lights over their respective levers. Once sequence accuracy was high and stable, the rats were divided into two groups. For the No-Lights group, the lights were eliminated without changing the response requirements. Sequence accuracy decreased in all subjects, but accuracy was higher than that predicted by random chance. More practice produced greater autonomy and reduced dependence on the guiding lights. For the Reversed-Lights group, the lights were presented in reversed order without changing the response requirements. Sequence accuracy immediately plummeted and did not recover, violating expectations of automatization. The guiding lights appeared to overshadow other sources of stimulus control.     

A neural network controller for the path tracking control of a hopping robot involving time delays

In this paper a hopping robot motion with offset mass is discussed. A mathematical model has been considered and an efficient single layered neural network has been developed to suit to the dynamics of the hopping robot, which ensures guaranteed tracking performance leading to the stability of the otherwise unstable system. The neural network takes advantage of the robot regressor dynamics that expresses the highly nonlinear robot dynamics in a linear form in terms of the known and unknown robot parameters. Time delays in the control mechanism play a vital role in the motion of hopping robots. The present work also enables us to estimate the maximum time delay admissible with out losing the guaranteed tracking performance. Further this neural network does not require offline training procedures. The salient features are highlighted by appropriate simulations.     

Development of feature detectors by self-organization

We present a two-layered network of linear neurons that organizes itself as to extract the complete information contained in a set of presented patterns. The weights between layers obey a Hebbian rule. We propose a local anti-Hebbian rule for lateral, hierarchically organized weights within the output layer. This rule forces the activities of the output units to become uncorrelated and the lateral weights to vanish. The weights between layers converge to the eigenvectors of the covariance matrix of input patterns, i.e., the network performs a principal component analysis, yielding all principal components. As a consequence of the proposed learning scheme, the output units become detectors of orthogonal features, similar to ones found in the brain of mammals.     

Model‐guided design of ligand‐regulated RNAi for programmable control of gene expression

Progress in constructing biological networks will rely on the development of more advanced components that can be predictably modified to yield optimal system performance. We have engineered an RNA‐based platform, which we call an shRNA switch, that provides for integrated ligand control of RNA interference (RNAi) by modular coupling of an aptamer, competing strand, and small hairpin (sh)RNA stem into a single component that links ligand concentration and target gene expression levels. A combined experimental and mathematical modelling approach identified multiple tuning strategies and moves towards a predictable framework for the forward design of shRNA switches. The utility of our platform is highlighted by the demonstration of fine‐tuning, multi‐input control, and model‐guided design of shRNA switches with an optimized dynamic range. Thus, shRNA switches can serve as an advanced component for the construction of complex biological systems and offer a controlled means of activating RNAi in disease therapeutics.     

Dynamic properties of two control systems underlying visually guided turning in house-flies

Summary The compensatory optomotor turning reaction as well as the turning response towards objects play an important role in visual orientation. On the basis of behavioural experiments under precisely defined stimulus conditions it is concluded that in female house-flies these motion-dependent responses are mediated by two parallel control systems with different dynamic and spatial integration properties. One of them (large-field system) is most sensitive to the motion of large textured patterns and controls the yaw torque mainly at low oscillation frequencies (below 0.1 Hz) of the stimulus panorama. In contrast, the other control system (small-field system) is tuned to the detection of relatively small moving patterns and shows its strongest responses at high oscillation frequencies (between 1 and 4 Hz), i.e. in a frequency range where the large-field system contributes to the turning response with only a relatively small gain.In free flight, house-flies do not curve smoothly but in sequences of rapid turns which induce retinal large-field motion of continually changing sign (Wagner 1986b). The dynamic properties of the large-field system might thus be interpreted as a simple strategy to almost eliminate the unwanted optomotor yaw torque induced by active self-motion. In contrast, the small-field system might still be operational under these conditions.     

Biologically inspired kinematic synergies enable linear balance control of a humanoid robot

Despite many efforts, balance control of humanoid robots in the presence of unforeseen external or internal forces has remained an unsolved problem. The difficulty of this problem is a consequence of the high dimensionality of the action space of a humanoid robot, due to its large number of degrees of freedom (joints), and of non-linearities in its kinematic chains. Biped biological organisms face similar difficulties, but have nevertheless solved this problem. Experimental data reveal that many biological organisms reduce the high dimensionality of their action space by generating movements through linear superposition of a rather small number of stereotypical combinations of simultaneous movements of many joints, to which we refer as kinematic synergies in this paper. We show that by constructing two suitable non-linear kinematic synergies for the lower part of the body of a humanoid robot, balance control can in fact be reduced to a linear control problem, at least in the case of relatively slow movements. We demonstrate for a variety of tasks that the humanoid robot HOAP-2 acquires through this approach the capability to balance dynamically against unforeseen disturbances that may arise from external forces or from manipulating unknown loads.