Distributed scalable multi-robot learning using particle swarm optimization

Designing effective behavioral controllers for mobile robots can be difficult and tedious; this process can be circumvented by using online learning techniques which allow robots to generate their own controllers online in an automated fashion. In multi-robot systems, robots operating in parallel can potentially learn at a much faster rate by sharing information amongst themselves. In this work, we use an adapted version of the Particle Swarm Optimization algorithm in order to accomplish distributed online robotic learning in groups of robots with access to only local information. The effectiveness of the learning technique on a benchmark task (generating high-performance obstacle avoidance behavior) is evaluated for robot groups of various sizes, with the maximum group size allowing each robot to individually contain and manage a single PSO particle. To increase the realism of the technique, different PSO neighborhoods based on limitations of real robotic communication are tested and compared in this scenario. We explore the effect of varying communication power for one of these communication-based PSO neighborhoods. To validate the effectiveness of these learning techniques, fully distributed online learning experiments are run using a group of 10 real robots, generating results which support the findings from our simulations.     

Self-organized acquisition of situated behaviors

Summary The paper aims at a systematic approach to the self-organization of behavior. It is rooted in the ideas of situated artificial intelligence and introduces situated behavior as the target for the self-organization procedure. Based on a quantitative measure of behavioral situatedness a learning dynamics is introduced which enables the controller to sustain the situatedness of the agent. The principle is demonstrated with Khepera robots in a number of different environmental conditions.     

Self-organisation and communication in groups of simulated and physical robots

In social insects, both self-organisation and communication play a crucial role for the accomplishment of many tasks at a collective level. Communication is performed with different modalities, which can be roughly classified into three classes: indirect (stigmergic) communication, direct interactions and direct communication. The use of stigmergic communication is predominant in social insects (e.g. the pheromone trails in ants), where, however, direct interactions (e.g. antennation in ants) and direct communication (e.g. the waggle dance in honey bees) can also be observed. Taking inspiration from insect societies, we present an experimental study of self-organising behaviours for a group of robots, which exploit communication to coordinate their activities. In particular, the robots are placed in an arena presenting holes and open borders, which they should avoid while moving coordinately. Artificial evolution is responsible for the synthesis in a simulated environment of the robot’s neural controllers, which are subsequently tested on physical robots. We study different communication strategies among the robots: no direct communication, handcrafted signalling and a completely evolved approach. We show that the latter is the most efficient, suggesting that artificial evolution can produce behaviours that are more adaptive than those obtained with conventional design methodologies. Moreover, we show that the evolved controllers produce a self-organising system that is robust enough to be tested on physical robots, notwithstanding the huge gap between simulation and reality.Electronic Supplementary Material Supplementary material is available for this article at     

Automatic synthesis of rulesets for programmable stochastic self-assembly of rotationally symmetric robotic modules

Programmable stochastic self-assembly of modular robots provides promising means to formation of structures at different scales. One way to address the design of dedicated control rulesets for self-assembling robotic modules is to leverage formalisms based on graph grammar. While these tools are powerful and allow for formal analysis of the resulting controllers, expressing the embodiment of the robotic modules and therefore the physical structure of assemblies of such modules is not readily possible with such formalisms. This typically results in inefficient representation of ruleset controllers and poses limitations on automatizing ruleset synthesis methods, requiring manual design or tuning of the rules before deployment on the robotic modules. In this work, we consider robotic modules endowed with identical latching connectors arranged in a rotationally symmetric configuration. We extend a grammar formalism based on graphs and propose a new encoding of the modules’ internal states. This allows for formulating formal methods capable of automatically deriving the rules based on the morphology of the robotic modules, in particular their number of connectors. The derived rules are directly applicable to robotic modules with no further tuning. In addition, we show that our method allows for a reduced complexity in the rulesets, a particularly welcome feature in the case of limited on-board storage, computation, and communication resources. In order to illustrate the application of our method, we extend two synthesis algorithms from the literature, namely Singleton and Linchpin, to automatically synthesize rules applicable to our resource-constrained robotic modules. In order to increase the prototyping speed and the thoroughness of the validation for the synthesis algorithms, we leverage two complementary simulation frameworks capturing the system at different levels of abstraction. Finally, employing the generated rulesets, we conduct experiments with our robotic platform to demonstrate several assemblies.     

Visual guidance based on optic flow: a biorobotic approach.

This paper addresses some basic questions as to how vision links up with action and serves to guide locomotion in both biological and artificial creatures. The thorough knowledge gained during the past five decades on insects' sensory-motor abilities and the neuronal substrates involved has provided us with a rich source of inspiration for designing tomorrow's self-guided vehicles and micro-vehicles, which will be able to cope with unforeseen events on the ground, under water, in the air, in space, on other planets, and inside the human body. Insects can teach us some useful tricks for designing agile autonomous robots. Since constructing a "biorobot" first requires exactly formulating the biological principles presumably involved, it gives us a unique opportunity of checking the soundness and robustness of these principles by bringing them face to face with the real physical world. "Biorobotics" therefore goes one step beyond computer simulation. It leads to experimenting with real physical robots which have to pass the stringent test of the real world. Biorobotics provide us with a new tool, which can help neurobiologists and neuroethologists to identify and investigate worthwhile issues in the field of sensory-motor control. Here we describe some of the visually guided terrestrial and aerial robots we have developed since 1985 on the basis of our biological findings. All these robots behave in response to the optic flow, i.e., they work by measuring the slip speed of the retinal image. Optic flow is sensed on-board by miniature electro-optical velocity sensors. The very principle of these sensors was based on studies in which we recorded the responses of single identified neurons to single photoreceptor stimulation in a model visual system: the fly's compound eye.     

A fuzzy-based reliability system for knowledge sharing between robots in P2P JXTA-overlay platform

The design of an efficient collaborative multi-robot framework that ensures the autonomy and the individual requirements of the involved robots is a very challenging task. This requires designing an efficient platform for inter-robot communication. P2P is a good approach to achieve this goal. P2P aims at making the communication ubiquitous thereby crossing the communication boundary and has many attractive features to use it as a platform for collaborative multi-robot environments. In this paper, we present our implemented P2P system based on JXTA Overlay. We use JXTA Overlay as a platform for robot collaboration and knowledge sharing. We also propose a fuzzy-based peer reliability system for JXTA-Overlay platform considering three parameters: Actual Behavior Criterion (ABC), Mutually Agreed Behavior (MAB) and Reputation (R). We evaluated the knowledge sharing system by many experiments and show that this system has a good performance and can be used successfully for knowledge sharing between robots. Also, we present some simulation results, which show the fuzzy-based peer reliability system has a good behavior and can successfully select the best peer candidate.     

From network-to-antibody robustness in a bio-inspired immune system

Behavioural robustness at antibody and immune network level is discussed. The robustness of the immune response that drives an autonomous mobile robot is examined with two computational experiments in the autonomous mobile robots trajectory generation context in unknown environments. The immune response is met based on the immune network metaphor for different low-level behaviours coordination. These behaviours are activated when a robot sense the appropriate conditions in the environment in relation to the network current state. Results are obtained over a case study in computer simulation as well as in laboratory experiments with a Khepera II microrobot. In this work, we develop a set of tests where such an immune response is externally perturbed at network or low-level behavioural modules to analyse the robust capacity of the system to unexpected perturbations. Emergence of robust behaviour and high-level immune response relates to the coupling between behavioural modules that are selectively engaged with the environment based on immune response. Experimental evidence leads discussions on a dynamical systems perspective of behavioural robustness in artificial immune systems that goes beyond the isolated immune network response.     

Robot control and the evolution of modular neurodynamics

Summary A modular approach to neural behavior control of autonomous robots is presented. It is based on the assumption that complex internal dynamics of recurrent neural networks can efficiently solve complex behavior tasks. For the development of appropriate neural control structures an evolutionary algorithm is introduced, which is able to generate neuromodules with specific functional properties, as well as the connectivity structure for a modular synthesis of such modules. This so called ENS ³-algorithm does not use genetic coding. It is primarily designed to develop size and connectivity structure of neuro-controllers. But at the same time it optimizes also parameters of individual networks like synaptic weights and bias terms. For demonstration, evolved networks for the control of miniature Khepera robots are presented. The aim is to develop robust controllers in the sense that neuro-controllers evolved in a simulator show comparably good behavior when loaded to a real robot acting in a physical environment. Discussed examples of such controllers generate obstacle avoidance and phototropic behaviors in non-trivial environments.     

Self-organized flocking in mobile robot swarms

In this paper, we study self-organized flocking in a swarm of mobile robots. We present Kobot, a mobile robot platform developed specifically for swarm robotic studies. We describe its infrared-based short range sensing system, capable of measuring the distance from obstacles and detecting kin robots, and a novel sensing system called the virtual heading system (VHS) which uses a digital compass and a wireless communication module for sensing the relative headings of neighboring robots. We propose a behavior based on heading alignment and proximal control that is capable of generating self-organized flocking in a swarm of Kobots. By self-organized flocking we mean that a swarm of mobile robots, initially connected via proximal sensing, is able to wander in an environment by moving as a coherent group in open space and to avoid obstacles as if it were a “super-organism”. We propose a number of metrics to evaluate the quality of flocking. We use a default set of behavioral parameter values that can generate acceptable flocking in robots, and analyze the sensitivity of the flocking behavior against changes in each of the parameters using the metrics that were proposed. We show that the proposed behavior can generate flocking in a small group of physical robots in a closed arena as well as in a swarm of 1000 simulated robots in open space. We vary the three main characteristics of the VHS, namely: (1) the amount and nature of noise in the measurement of heading, (2) the number of VHS neighbors, and (3) the range of wireless communication. Our experiments show that the range of communication is the main factor that determines the maximum number of robots that can flock together and that the behavior is highly robust against the other two VHS characteristics. We conclude by discussing this result in the light of related theoretical studies in statistical physics.     

Bio-Inspired Controller for a Robot Cheetah with a Neural Mechanism Controlling Leg Muscles

The realization of a high-speed running robot is one of the most challenging problems in developing legged robots.The excellent performance of cheetahs provides inspiration for the control and mechanical design of such robots.This paper presents a three-dimensional model of a cheetah that predicts the locomotory behaviors of a running cheetah.Applying biological knowledge of the neural mechanism,we control the muscle flexion and extension during the stance phase,and control the positions of the joints in the flight phase via a PD controller to minimize complexity.The proposed control strategy is shown to achieve similar locomotion of a real cheetah.The simulation realizes good biological properties,such as the leg retraction,ground reaction force,and spring-like leg behavior.The stable bounding results show the promise of the controller in high-speed locomotion.The model can reach 2.7 m·s- 1 as the highest speed,and can accelerate from 0 to 1.5 m·s -1 in one stride cycle.A mechanical structure based on this simulation is designed to demonstrate the control approach,and the most recently developed hindlimb controlled by the proposed controller is presented in swinging-leg experiments and jump-force experiments.     

Massively multi-robot simulation in stage

Stage is a C++ software library that simulates multiple mobile robots. Stage version 2, as the simulation backend for the Player/Stage system, may be the most commonly used robot simulator in research and university teaching today. Development of Stage version 3 has focused on improving scalability, usability, and portability. This paper examines Stage’s scalability. We propose a simple benchmark for multi-robot simulator performance, and present results for Stage. Run time is shown to scale approximately linearly with population size up to 100,000 robots. For example, Stage simulates 1 simple robot at around 1,000 times faster than real time, and 1,000 simple robots at around real time. These results suggest that Stage may be useful for swarm robotics researchers who would otherwise use custom simulators, with their attendant disadvantages in terms of code reuse and transparency.     

What does it take to evolve behaviorally complex organisms?

What genotypic features explain the evolvability of organisms that have to accomplish many different tasks? The genotype of behaviorally complex organisms may be more likely to encode modular neural architectures because neural modules dedicated to distinct tasks avoid neural interference, i.e. the arrival of conflicting messages for changing the value of connection weights during learning. However, if the connection weights for the various modules are genetically inherited, this raises the problem of genetic linkage: favorable mutations may fall on one portion of the genotype encoding one neural module and unfavorable mutations on another portion encoding another module. We show that this can prevent the genotype from reaching an adaptive optimum. This effect is different from other linkage effects described in the literature and we argue that it represents a new class of genetic constraints. Using simulations we show that sexual reproduction can alleviate the problem of genetic linkage by recombining separate modules all of which incorporate either favorable or unfavorable mutations. We speculate that this effect may contribute to the taxonomic prevalence of sexual reproduction among higher organisms. In addition to sexual recombination, the problem of genetic linkage for behaviorally complex organisms may be mitigated by entrusting evolution with the task of finding appropriate modular architectures and learning with the task of finding the appropriate connection weights for these architectures.     

Collective unary decision-making by decentralized multiple-robot systems applied to the task-sequencing problem

When a complex mission must be undertaken, it often can be simplified by dividing it into a sequence of smaller subtasks, which are then completed in order. This strategy implicitly requires a system to recognize the completion of each subtask and make the decision to begin work on the next one. Decentralized multiple-robot systems can tackle many tasks, but their behavior is typified by continuous responses to stimuli. Task sequencing, however, demands a controlled, self-induced phase change in collective behavior—working on one task one moment and then on a different task the next—which is nontrivial for an emergent system. The main contribution of this study is a collective decision-making framework for decentralized multiple-robot systems that enables such a system to cooperatively decide that a current task has been completed and thus focus its attention on the next one in a sequence using only anonymous local communication. Central to the framework is the use of consensus, whereby task sequencing is delayed until a prespecified proportion of a system’s robots agree that the current task is complete, reducing the likelihood of premature decisions. Two low-cost consensus estimation strategies are presented, both of which are practical for the extremely simple robots that are expected to compose large decentralized systems. Experiments in simulation and with real robots demonstrate that the proposed decision-making framework performs as predicted. Although the specific application of collective decision-making in this work is the cooperative task-sequencing problem, the proposed decision-making framework potentially has many additional applications.     

Designing pheromone communication in swarm robotics: Group foraging behavior mediated by chemical substance

In swarm robotics, communication among the robots is essential. Inspired by biological swarms using pheromones, we propose the use of chemical compounds to realize group foraging behavior in robot swarms. We designed a fully autonomous robot, and then created a swarm using ethanol as the trail pheromone allowing the robots to communicate with one another indirectly via pheromone trails. Our group recruitment and cooperative transport algorithms provide the robots with the required swarm behavior. We conducted both simulations and experiments with real robot swarms, and analyzed the data statistically to investigate any changes caused by pheromone communication in the performance of the swarm in solving foraging recruitment and cooperative transport tasks. The results show that the robots can communicate using pheromone trails, and that the improvement due to pheromone communication may be non-linear, depending on the size of the robot swarm.     

Self-organized cooperation between robotic swarms

We study self-organized cooperation between heterogeneous robotic swarms. The robots of each swarm play distinct roles based on their different characteristics. We investigate how the use of simple local interactions between the robots of the different swarms can let the swarms cooperate in order to solve complex tasks. We focus on an indoor navigation task, in which we use a swarm of wheeled robots, called foot-bots, and a swarm of flying robots that can attach to the ceiling, called eye-bots. The task of the foot-bots is to move back and forth between a source and a target location. The role of the eye-bots is to guide foot-bots: they choose positions at the ceiling and from there give local directional instructions to foot-bots passing by. To obtain efficient paths for foot-bot navigation, eye-bots need on the one hand to choose good positions and on the other hand learn the right instructions to give. We investigate each of these aspects. Our solution is based on a process of mutual adaptation, in which foot-bots execute instructions given by eye-bots, and eye-bots observe the behavior of foot-bots to adapt their position and the instructions they give. Our approach is inspired by pheromone mediated navigation of ants, as eye-bots serve as stigmergic markers for foot-bot navigation. Through simulation, we show how this system is able to find efficient paths in complex environments, and to display different kinds of complex and scalable self-organized behaviors, such as shortest path finding and automatic traffic spreading.     

Evolution of Collective Behaviors for a Real Swarm of Aquatic Surface Robots

Swarm robotics is a promising approach for the coordination of large numbers of robots. While previous studies have shown that evolutionary robotics techniques can be applied to obtain robust and efficient self-organized behaviors for robot swarms, most studies have been conducted in simulation, and the few that have been conducted on real robots have been confined to laboratory environments. In this paper, we demonstrate for the first time a swarm robotics system with evolved control successfully operating in a real and uncontrolled environment. We evolve neural network-based controllers in simulation for canonical swarm robotics tasks, namely homing, dispersion, clustering, and monitoring. We then assess the performance of the controllers on a real swarm of up to ten aquatic surface robots. Our results show that the evolved controllers transfer successfully to real robots and achieve a performance similar to the performance obtained in simulation. We validate that the evolved controllers display key properties of swarm intelligence-based control, namely scalability, flexibility, and robustness on the real swarm. We conclude with a proof-of-concept experiment in which the swarm performs a complete environmental monitoring task by combining multiple evolved controllers.     

In silico evolution of functional modules in biochemical networks

Understanding the large reaction networks found in biological systems is a daunting task. One approach is to divide a network into more manageable smaller modules, thus simplifying the problem. This is a common strategy used in engineering. However, the process of identifying biological modules is still in its infancy and very little is understood about the range and capabilities of motif structures found in biological modules. In order to delineate these modules, a library of functional motifs has been generated via in silico evolution techniques. On the basis of their functional forms, networks were evolved from four broad areas: oscillators, bistable switches, homeostatic systems and frequency filters. Some of these motifs were constructed from simple mass action kinetics, others were based on Michaelis-Menten kinetics as found in protein/protein networks and the remainder were based on Hill equations as found in gene/protein interaction networks. The purpose of the study is to explore the capabilities of different network architectures and the rich variety of functional forms that can be generated. Ultimately, the library may be used to delineate functional motifs in real biological networks.     

Speciation of chromium in plasma and liver tissue of endstage renal failure patients on continuous ambulatory peritoneal dialysis

This work is a first attempt to determine the speciation of Cr in human plasma. With the aid of in vitro and in vivo⁵¹Cr-labeled experiments, it was possible to develop the necessary biochemical techniques for the separation of the plasma proteins. Further work will use real samples, taking care to avoid contamination of the various fractions and to preserve the original binding of the Cr to the specific plasma compounds. In a first attempt on the distribution of Cr over the different organelles of liver tissue, work will be restricted to in vivo labeled experiments with rats. The procedure to do the speciation work seems so elaborate that it may be impossible ever to achieve the contemplated speciation of Cr in human liver tissue by subcellular fractionation.     

Haptic/Graphic Rehabilitation: Integrating a Robot into a Virtual Environment Library and Applying it to Stroke Therapy

Recent research that tests interactive devices for prolonged therapy practice has revealed new prospects for robotics combined with graphical and other forms of biofeedback. Previous human-robot interactive systems have required different software commands to be implemented for each robot leading to unnecessary developmental overhead time each time a new system becomes available. For example, when a haptic/graphic virtual reality environment has been coded for one specific robot to provide haptic feedback, that specific robot would not be able to be traded for another robot without recoding the program. However, recent efforts in the open source community have proposed a wrapper class approach that can elicit nearly identical responses regardless of the robot used. The result can lead researchers across the globe to perform similar experiments using shared code. Therefore modular "switching out"of one robot for another would not affect development time. In this paper, we outline the successful creation and implementation of a wrapper class for one robot into the open-source H3DAPI, which integrates the software commands most commonly used by all robots.     

A patch-based method for repetitive and transient event detection in fluorescence imaging

Automatic detection and characterization of molecular behavior in large data sets obtained by fast imaging in advanced light microscopy become key issues to decipher the dynamic architectures and their coordination in the living cell. Automatic quantification of the number of sudden and transient events observed in fluorescence microscopy is discussed in this paper. We propose a calibrated method based on the comparison of image patches expected to distinguish sudden appearing/vanishing fluorescent spots from other motion behaviors such as lateral movements. We analyze the performances of two statistical control procedures and compare the proposed approach to a frame difference approach using the same controls on a benchmark of synthetic image sequences. We have then selected a molecular model related to membrane trafficking and considered real image sequences obtained in cells stably expressing an endocytic-recycling trans-membrane protein, the Langerin-YFP, for validation. With this model, we targeted the efficient detection of fast and transient local fluorescence concentration arising in image sequences from a data base provided by two different microscopy modalities, wide field (WF) video microscopy using maximum intensity projection along the axial direction and total internal reflection fluorescence microscopy. Finally, the proposed detection method is briefly used to statistically explore the effect of several perturbations on the rate of transient events detected on the pilot biological model.