Closed-loop interactions between a shoal of zebrafish and a group of robotic fish in a circular corridor

Collective behavior based on self-organization has been observed in populations of animals from insects to vertebrates. These findings have motivated engineers to investigate approaches to control autonomous multi-robot systems able to reproduce collective animal behaviors, and even to collectively interact with groups of animals. In this article, we show collective decision making by a group of autonomous robots and a group of zebrafish, leading to a shared decision about swimming direction. The robots can also modulate the collective decision-making process in biased and non-biased experimental setups. These results demonstrate the possibility of creating mixed societies of vertebrates and robots in order to study or control animal behavior.     

Self-organized acquisition of situated behaviors

Summary The paper aims at a systematic approach to the self-organization of behavior. It is rooted in the ideas of situated artificial intelligence and introduces situated behavior as the target for the self-organization procedure. Based on a quantitative measure of behavioral situatedness a learning dynamics is introduced which enables the controller to sustain the situatedness of the agent. The principle is demonstrated with Khepera robots in a number of different environmental conditions.     

Self-organized flocking in mobile robot swarms

In this paper, we study self-organized flocking in a swarm of mobile robots. We present Kobot, a mobile robot platform developed specifically for swarm robotic studies. We describe its infrared-based short range sensing system, capable of measuring the distance from obstacles and detecting kin robots, and a novel sensing system called the virtual heading system (VHS) which uses a digital compass and a wireless communication module for sensing the relative headings of neighboring robots. We propose a behavior based on heading alignment and proximal control that is capable of generating self-organized flocking in a swarm of Kobots. By self-organized flocking we mean that a swarm of mobile robots, initially connected via proximal sensing, is able to wander in an environment by moving as a coherent group in open space and to avoid obstacles as if it were a “super-organism”. We propose a number of metrics to evaluate the quality of flocking. We use a default set of behavioral parameter values that can generate acceptable flocking in robots, and analyze the sensitivity of the flocking behavior against changes in each of the parameters using the metrics that were proposed. We show that the proposed behavior can generate flocking in a small group of physical robots in a closed arena as well as in a swarm of 1000 simulated robots in open space. We vary the three main characteristics of the VHS, namely: (1) the amount and nature of noise in the measurement of heading, (2) the number of VHS neighbors, and (3) the range of wireless communication. Our experiments show that the range of communication is the main factor that determines the maximum number of robots that can flock together and that the behavior is highly robust against the other two VHS characteristics. We conclude by discussing this result in the light of related theoretical studies in statistical physics.     

Task partitioning in swarms of robots: an adaptive method for strategy selection

Task partitioning is the decomposition of a task into two or more sub-tasks that can be tackled separately. Task partitioning can be observed in many species of social insects, as it is often an advantageous way of organizing the work of a group of individuals. Potential advantages of task partitioning are, among others: reduction of interference between workers, exploitation of individuals?? skills and specializations, energy efficiency, and higher parallelism. Even though swarms of robots can benefit from task partitioning in the same way as social insects do, only few works in swarm robotics are dedicated to this subject. In this paper, we study the case in which a swarm of robots has to tackle a task that can be partitioned into a sequence of two sub-tasks. We propose a method that allows the individual robots in the swarm to decide whether to partition the given task or not. The method is self-organized, relies on the experience of each individual, and does not require explicit communication between robots. We evaluate the method in simulation experiments, using foraging as testbed. We study cases in which task partitioning is preferable and cases in which it is not. We show that the proposed method leads to good performance of the swarm in both cases, by employing task partitioning only when it is advantageous. We also show that the swarm is able to react to changes in the environmental conditions by adapting the behavior on-line. Scalability experiments show that the proposed method performs well across all the tested group sizes.     

Coevolution of Quantum and Classical Strategies on Evolving Random Networks

We study the coevolution of quantum and classical strategies on weighted and directed random networks in the realm of the prisoner’s dilemma game. During the evolution, agents can break and rewire their links with the aim of maximizing payoffs, and they can also adjust the weights to indicate preferences, either positive or negative, towards their neighbors. The network structure itself is thus also subject to evolution. Importantly, the directionality of links does not affect the accumulation of payoffs nor the strategy transfers, but serves only to designate the owner of each particular link and with it the right to adjust the link as needed. We show that quantum strategies outperform classical strategies, and that the critical temptation to defect at which cooperative behavior can be maintained rises, if the network structure is updated frequently. Punishing neighbors by reducing the weights of their links also plays an important role in maintaining cooperation under adverse conditions. We find that the self-organization of the initially random network structure, driven by the evolutionary competition between quantum and classical strategies, leads to the spontaneous emergence of small average path length and a large clustering coefficient.     

Self-organization and productivity in semi-arid ecosystems: implications of seasonality in rainfall

Spatial self-organization including striking vegetation patterns observed in arid ecosystems has been studied in models with uniform rainfall. In this paper, we present a fully seasonal rainfall model that produces vegetation patterns found in nature by including the natural adaptation of plants to scarcity of water and the consequent seasonal variation in their growth and metabolic rate. We present results for the mean-field and spatially extended versions of the model. We find that the patterns depend on the duration of the wet season even with fixed total annual precipitation (PPT) showing how seasonality affects spatial self-organization. We observe that the productivity can vary for fixed PPT as a function of the duration thereby providing another source of observed variations. We compute the maximum vegetation cover as function of PPT and find that the behavior is consistent with observations. We comment on the implications for regime shifts due to increased interannual fluctuations caused by climatic changes. Our specific model calculations provide more general conclusions for ecosystems with competition for scarce resources due to seasonal variations in the resource, especially for self-organization and productivity.     

Designing pheromone communication in swarm robotics: Group foraging behavior mediated by chemical substance

In swarm robotics, communication among the robots is essential. Inspired by biological swarms using pheromones, we propose the use of chemical compounds to realize group foraging behavior in robot swarms. We designed a fully autonomous robot, and then created a swarm using ethanol as the trail pheromone allowing the robots to communicate with one another indirectly via pheromone trails. Our group recruitment and cooperative transport algorithms provide the robots with the required swarm behavior. We conducted both simulations and experiments with real robot swarms, and analyzed the data statistically to investigate any changes caused by pheromone communication in the performance of the swarm in solving foraging recruitment and cooperative transport tasks. The results show that the robots can communicate using pheromone trails, and that the improvement due to pheromone communication may be non-linear, depending on the size of the robot swarm.     

Bio-inspired design strategies for central pattern generator control in modular robotics

New findings in the nervous system of invertebrates have shown how a number of features of central pattern generator (CPG) circuits contribute to the generation of robust flexible rhythms. In this paper we consider recently revealed strategies that living CPGs follow to design CPG control paradigms for modular robots. To illustrate them, we divide the task of designing an example CPG for a modular robot into independent problems. We formulate each problem in a general way and provide a bio-inspired solution for each of them: locomotion information coding, individual module control and inter-module coordination. We analyse the stability of the CPG numerically, and then test it on a real robot. We analyse steady state locomotion and recovery after perturbations. In both cases, the robot is able to autonomously find a stable effective locomotion state. Finally, we discuss how these strategies can result in a more general design approach for CPG-based locomotion.     

Modelling foraging ants in a dynamic and confined environment

In social insects, the superposition of simple individual behavioral rules leads to the emergence of complex collective patterns and helps solve difficult problems inherent to surviving in hostile habitats. Modelling ant colony foraging reveals strategies arising from the insects’ self-organization and helps develop of new computational strategies in order to solve complex problems. This paper presents advances in modelling ants’ behavior when foraging in a confined and dynamic environment, based on experiments with the Argentine ant Linepithema humile in a relatively complex artificial network. We propose a model which overcomes the problem of stagnation observed in earlier models by taking into account additional biological aspects, by using non-linear functions for the deposit, perception and evaporation of pheromone, and by introducing new mechanisms to represent randomness and the exploratory behavior of the ants.     

Using robots to understand social behaviour

A major challenge in studying social behaviour stems from the need to disentangle the behaviour of each individual from the resulting collective. One way to overcome this problem is to construct a model of the behaviour of an individual, and observe whether combining many such individuals leads to the predicted outcome. This can be achieved by using robots. In this review we discuss the strengths and weaknesses of such an approach for studies of social behaviour. We find that robots—whether studied in groups of simulated or physical robots, or used to infiltrate and manipulate groups of living organisms—have important advantages over conventional individual‐based models and have contributed greatly to the study of social behaviour. In particular, robots have increased our understanding of self‐organization and the evolution of cooperative behaviour and communication. However, the resulting findings have not had the desired impact on the biological community. We suggest reasons for why this may be the case, and how the benefits of using robots can be maximized in future research on social behaviour.     

Local force cues for strength and stability in a distributed robotic construction system

Construction of spatially extended, self-supporting structures requires a consideration of structural stability throughout the building sequence. For collective construction systems, where independent agents act with variable order and timing under decentralized control, ensuring stability is a particularly pronounced challenge. Previous research in this area has largely neglected considering stability during the building process. Physical forces present throughout a structure may be usable as a cue to inform agent actions as well as an indirect communication mechanism (stigmergy) to coordinate their behavior, as adding material leads to redistribution of forces which then informs the addition of further material. Here we consider in simulation a system of decentralized climbing robots capable of traversing and extending a two-dimensional truss structure, and explore the use of feedback based on force sensing as a way for the swarm to anticipate and prevent structural failures. We consider a scenario in which robots are tasked with building an unsupported cantilever across a gap, as for a bridge, where the goal is for the swarm to build any stable spanning structure rather than to construct a specific predetermined blueprint. We show that access to local force measurements enables robots to build cantilevers that span significantly farther than those built by robots without access to such information. This improvement is achieved by taking measures to maintain both strength and stability, where strength is ensured by paying attention to forces during locomotion to prevent joints from breaking, and stability is maintained by looking at how loads transfer to the ground to ensure against toppling. We show that swarms that take both kinds of forces into account have improved building performance, in both structured settings with flat ground and unpredictable environments with rough terrain.     

A Robotic Approach to Understanding the Role and the Mechanism of Vicarious Trial-And-Error in a T-Maze Task

Vicarious trial-and-error (VTE) is a behavior observed in rat experiments that seems to suggest self-conflict. This behavior is seen mainly when the rats are uncertain about making a decision. The presence of VTE is regarded as an indicator of a deliberative decision-making process, that is, searching, predicting, and evaluating outcomes. This process is slower than automated decision-making processes, such as reflex or habituation, but it allows for flexible and ongoing control of behavior. In this study, we propose for the first time a robotic model of VTE to see if VTE can emerge just from a body-environment interaction and to show the underlying mechanism responsible for the observation of VTE and the advantages provided by it. We tried several robots with different parameters, and we have found that they showed three different types of VTE: high numbers of VTE at the beginning of learning, decreasing numbers afterward (similar VTE pattern to experiments with rats), low during the whole learning period, and high numbers all the time. Therefore, we were able to reproduce the phenomenon of VTE in a model robot using only a simple dynamical neural network with Hebbian learning, which suggests that VTE is an emergent property of a plastic and embodied neural network. From a comparison of the three types of VTE, we demonstrated that 1) VTE is associated with chaotic activity of neurons in our model and 2) VTE-showing robots were robust to environmental perturbations. We suggest that the instability of neuronal activity found in VTE allows ongoing learning to rebuild its strategy continuously, which creates robust behavior. Based on these results, we suggest that VTE is caused by a similar mechanism in biology and leads to robust decision making in an analogous way.     

Task-Level Strategies for Human Sagittal-Plane Running Maneuvers Are Consistent with Robotic Control Policies

The strategies that humans use to control unsteady locomotion are not well understood. A “spring-mass” template comprised of a point mass bouncing on a sprung leg can approximate both center of mass movements and ground reaction forces during running in humans and other animals. Legged robots that operate as bouncing, “spring-mass” systems can maintain stable motion using relatively simple, distributed feedback rules. We tested whether the changes to sagittal-plane movements during five running tasks involving active changes to running height, speed, and orientation were consistent with the rules used by bouncing robots to maintain stability. Changes to running height were associated with changes to leg force but not stance duration. To change speed, humans primarily used a “pogo stick” strategy, where speed changes were associated with adjustments to fore-aft foot placement, and not a “unicycle” strategy involving systematic changes to stance leg hip moment. However, hip moments were related to changes to body orientation and angular speed. Hip moments could be described with first order proportional-derivative relationship to trunk pitch. Overall, the task-level strategies used for body control in humans were consistent with the strategies employed by bouncing robots. Identification of these behavioral strategies could lead to a better understanding of the sensorimotor mechanisms that allow for effective unsteady locomotion.     

Interactive robots in experimental biology

Interactive robots have the potential to revolutionise the study of social behaviour because they provide several methodological advances. In interactions with live animals, the behaviour of robots can be standardised, morphology and behaviour can be decoupled (so that different morphologies and behavioural strategies can be combined), behaviour can be manipulated in complex interaction sequences and models of behaviour can be embodied by the robot and thereby be tested. Furthermore, robots can be used as demonstrators in experiments on social learning. As we discuss here, the opportunities that robots create for new experimental approaches have far-reaching consequences for research in fields such as mate choice, cooperation, social learning, personality studies and collective behaviour.     

Artificial pheromone for path selection by a foraging swarm of robots

Foraging robots involved in a search and retrieval task may create paths to navigate faster in their environment. In this context, a swarm of robots that has found several resources and created different paths may benefit strongly from path selection. Path selection enhances the foraging behavior by allowing the swarm to focus on the most profitable resource with the possibility for unused robots to stop participating in the path maintenance and to switch to another task. In order to achieve path selection, we implement virtual ants that lay artificial pheromone inside a network of robots. Virtual ants are local messages transmitted by robots; they travel along chains of robots and deposit artificial pheromone on the robots that are literally forming the chain and indicating the path. The concentration of artificial pheromone on the robots allows them to decide whether they are part of a selected path. We parameterize the mechanism with a mathematical model and provide an experimental validation using a swarm of 20 real robots. We show that our mechanism favors the selection of the closest resource is able to select a new path if a selected resource becomes unavailable and selects a newly detected and better resource when possible. As robots use very simple messages and behaviors, the system would be particularly well suited for swarms of microrobots with minimal abilities.     

From embodied mind to embodied robotics: Humanities and system theoretical aspects

After an introduction (1) the article analyzes the evolution of the embodied mind (2), the innovation of embodied robotics (3), and finally discusses conclusions of embodied robotics for human responsibility (4). Considering the evolution of the embodied mind (2), we start with an introduction of complex systems and nonlinear dynamics (2.1), apply this approach to neural self-organization (2.2), distinguish degrees of complexity of the brain (2.3), explain the emergence of cognitive states by complex systems dynamics (2.4), and discuss criteria for modeling the brain as complex nonlinear system (2.5). The innovation of embodied robotics (3) is a challenge of future technology. We start with the distinction of symbolic and embodied AI (3.1) and explain embodied robots as dynamical systems (3.2). Self-organization needs self-control of technical systems (3.3). Cellular neural networks (CNN) are an example of self-organizing technical systems offering new avenues for neurobionics (3.4). In general, technical neural networks support different kinds of learning robots (3.5). Finally, embodied robotics aim at the development of cognitive and conscious robots (3.6).     

Distributed scalable multi-robot learning using particle swarm optimization

Designing effective behavioral controllers for mobile robots can be difficult and tedious; this process can be circumvented by using online learning techniques which allow robots to generate their own controllers online in an automated fashion. In multi-robot systems, robots operating in parallel can potentially learn at a much faster rate by sharing information amongst themselves. In this work, we use an adapted version of the Particle Swarm Optimization algorithm in order to accomplish distributed online robotic learning in groups of robots with access to only local information. The effectiveness of the learning technique on a benchmark task (generating high-performance obstacle avoidance behavior) is evaluated for robot groups of various sizes, with the maximum group size allowing each robot to individually contain and manage a single PSO particle. To increase the realism of the technique, different PSO neighborhoods based on limitations of real robotic communication are tested and compared in this scenario. We explore the effect of varying communication power for one of these communication-based PSO neighborhoods. To validate the effectiveness of these learning techniques, fully distributed online learning experiments are run using a group of 10 real robots, generating results which support the findings from our simulations.     

Behavior reuse and virtual sensors in the evolution of complex behavior architectures

Summary In this paper we discuss some of the new work we have been carrying out with the objective of making evolutionarily obtained behavior based architectures and modules for autonomous robots more standardized and interchangeable. The architectures contemplated here are based on a multiple behavior structure where all of the modules, as well as their interconnections, are automatically obtained through evolutionary processes. The main objective of this line of research is to obtain procedures that permit producing behavior based controllers that work on real robots operating in real environments as independently of the platform as possible. In this particular paper we will concentrate on different aspects regarding the inclusion of virtual sensors as a way to make improved use of the capabilities of the different platforms and on the reuse of behavior modules. This reuse will be contemplated within the same behavioral architecture and from the point of view of transferring behavior modules from one platform to a different one.     

Cooperative object transport with a swarm of e-puck robots: robustness and scalability of evolved collective strategies

Cooperative object transport in distributed multi-robot systems requires the coordination and synchronisation of pushing/pulling forces by a group of autonomous robots in order to transport items that cannot be transported by a single agent. The results of this study show that fairly robust and scalable collective transport strategies can be generated by robots equipped with a relatively simple sensory apparatus (i.e. no force sensors and no devices for direct communication). In the experiments described in this paper, homogeneous groups of physical e-puck robots are required to coordinate and synchronise their actions in order to transport a heavy rectangular cuboid object as far as possible from its starting position to an arbitrary direction. The robots are controlled by dynamic neural networks synthesised using evolutionary computation techniques. The best evolved controller demonstrates an effective group transport strategy that is robust to variability in the physical characteristics of the object (i.e. object mass and size of the longest object’s side) and scalable to different group sizes. To run these experiments, we designed, built, and mounted on the robots a new sensor that returns the agents’ displacement on a 2D plane. The study shows that the feedback generated by the robots’ sensors relative to the object’s movement is sufficient to allow the robots to coordinate their efforts and to sustain the transports for an extended period of time. By extensively analysing successful behavioural strategies, we illustrate the nature of the operational mechanisms underpinning the coordination and synchronisation of actions during group transport.     

Emergence of Polymorphic Mating Strategies in Robot Colonies

Polymorphism has fascinated evolutionary biologists since the time of Darwin. Biologists have observed discrete alternative mating strategies in many different species. In this study, we demonstrate that polymorphic mating strategies can emerge in a colony of hermaphrodite robots. We used a survival and reproduction task where the robots maintained their energy levels by capturing energy sources and physically exchanged genotypes for the reproduction of offspring. The reproductive success was dependent on the individuals'' energy levels, which created a natural trade-off between the time invested in maintaining a high energy level and the time invested in attracting mating partners. We performed experiments in environments with different density of energy sources and observed a variety in the mating behavior when a robot could see both an energy source and a potential mating partner. The individuals could be classified into two phenotypes: 1) forager, who always chooses to capture energy sources, and 2) tracker, who keeps track of potential mating partners if its energy level is above a threshold. In four out of the seven highest fitness populations in different environments, we found subpopulations with distinct differences in genotype and in behavioral phenotype. We analyzed the fitnesses of the foragers and the trackers by sampling them from each subpopulation and mixing with different ratios in a population. The fitness curves for the two subpopulations crossed at about 25% of foragers in the population, showing the evolutionary stability of the polymorphism. In one of those polymorphic populations, the trackers were further split into two subpopulations: (strong trackers) and (weak trackers). Our analyses show that the population consisting of three phenotypes also constituted several stable polymorphic evolutionarily stable states. To our knowledge, our study is the first to demonstrate the emergence of polymorphic evolutionarily stable strategies within a robot evolution framework.