Deriving neural network controllers from neuro-biological data: implementation of a single-leg stick insect controller

This article presents modular recurrent neural network controllers for single legs of a biomimetic six-legged robot equipped with standard DC motors. Following arguments of Ekeberg et?al. (Arthropod Struct Dev 33:287?C300, 2004), completely decentralized and sensori-driven neuro-controllers were derived from neuro-biological data of stick-insects. Parameters of the controllers were either hand-tuned or optimized by an evolutionary algorithm. Employing identical controller structures, qualitatively similar behaviors were achieved for robot and for stick insect simulations. For a wide range of perturbing conditions, as for instance changing ground height or up- and downhill walking, swing as well as stance control were shown to be robust. Behavioral adaptations, like varying locomotion speeds, could be achieved by changes in neural parameters as well as by a mechanical coupling to the environment. To a large extent the simulated walking behavior matched biological data. For example, this was the case for body support force profiles and swing trajectories under varying ground heights. The results suggest that the single-leg controllers are suitable as modules for hexapod controllers, and they might therefore bridge morphological- and behavioral-based approaches to stick insect locomotion control.     

Evolution of Collective Behaviors for a Real Swarm of Aquatic Surface Robots

Swarm robotics is a promising approach for the coordination of large numbers of robots. While previous studies have shown that evolutionary robotics techniques can be applied to obtain robust and efficient self-organized behaviors for robot swarms, most studies have been conducted in simulation, and the few that have been conducted on real robots have been confined to laboratory environments. In this paper, we demonstrate for the first time a swarm robotics system with evolved control successfully operating in a real and uncontrolled environment. We evolve neural network-based controllers in simulation for canonical swarm robotics tasks, namely homing, dispersion, clustering, and monitoring. We then assess the performance of the controllers on a real swarm of up to ten aquatic surface robots. Our results show that the evolved controllers transfer successfully to real robots and achieve a performance similar to the performance obtained in simulation. We validate that the evolved controllers display key properties of swarm intelligence-based control, namely scalability, flexibility, and robustness on the real swarm. We conclude with a proof-of-concept experiment in which the swarm performs a complete environmental monitoring task by combining multiple evolved controllers.     

Self-organisation and communication in groups of simulated and physical robots

In social insects, both self-organisation and communication play a crucial role for the accomplishment of many tasks at a collective level. Communication is performed with different modalities, which can be roughly classified into three classes: indirect (stigmergic) communication, direct interactions and direct communication. The use of stigmergic communication is predominant in social insects (e.g. the pheromone trails in ants), where, however, direct interactions (e.g. antennation in ants) and direct communication (e.g. the waggle dance in honey bees) can also be observed. Taking inspiration from insect societies, we present an experimental study of self-organising behaviours for a group of robots, which exploit communication to coordinate their activities. In particular, the robots are placed in an arena presenting holes and open borders, which they should avoid while moving coordinately. Artificial evolution is responsible for the synthesis in a simulated environment of the robot’s neural controllers, which are subsequently tested on physical robots. We study different communication strategies among the robots: no direct communication, handcrafted signalling and a completely evolved approach. We show that the latter is the most efficient, suggesting that artificial evolution can produce behaviours that are more adaptive than those obtained with conventional design methodologies. Moreover, we show that the evolved controllers produce a self-organising system that is robust enough to be tested on physical robots, notwithstanding the huge gap between simulation and reality.Electronic Supplementary Material Supplementary material is available for this article at     

Integrating behavioral and neural data in a model of zebrafish network interaction

The spinal neural networks of larval zebrafish (Danio rerio) generate a variety of movements such as escape, struggling, and swimming. Various mechanisms at the neural and network levels have been proposed to account for switches between these behaviors. However, there are currently no detailed demonstrations of such mechanisms. This makes determining which mechanisms are plausible extremely difficult. In this paper, we propose a detailed biologically plausible model of the interactions between the swimming and escape networks in the larval zebrafish, while taking into account anatomical and physiological evidence. We show that the results of our neural model generate the expected behavior when used to control a hydrodynamic model of carangiform locomotion. As a result, the model presented here is a clear demonstration of a plausible mechanism by which these distinct behaviors can be controlled. Interestingly, the networks are anatomically overlapping, despite clear differences in behavioral function and physiology.     

Trade-off between Multiple Constraints Enables Simultaneous Formation of Modules and Hubs in Neural Systems

The formation of the complex network architecture of neural systems is subject to multiple structural and functional constraints. Two obvious but apparently contradictory constraints are low wiring cost and high processing efficiency, characterized by short overall wiring length and a small average number of processing steps, respectively. Growing evidence shows that neural networks are results from a trade-off between physical cost and functional value of the topology. However, the relationship between these competing constraints and complex topology is not well understood quantitatively. We explored this relationship systematically by reconstructing two known neural networks, Macaque cortical connectivity and C. elegans neuronal connections, from combinatory optimization of wiring cost and processing efficiency constraints, using a control parameter , and comparing the reconstructed networks to the real networks. We found that in both neural systems, the reconstructed networks derived from the two constraints can reveal some important relations between the spatial layout of nodes and the topological connectivity, and match several properties of the real networks. The reconstructed and real networks had a similar modular organization in a broad range of , resulting from spatial clustering of network nodes. Hubs emerged due to the competition of the two constraints, and their positions were close to, and partly coincided, with the real hubs in a range of values. The degree of nodes was correlated with the density of nodes in their spatial neighborhood in both reconstructed and real networks. Generally, the rebuilt network matched a significant portion of real links, especially short-distant ones. These findings provide clear evidence to support the hypothesis of trade-off between multiple constraints on brain networks. The two constraints of wiring cost and processing efficiency, however, cannot explain all salient features in the real networks. The discrepancy suggests that there are further relevant factors that are not yet captured here.     

Nonlinear system identification and control based on modular neural networks

A new approach for nonlinear system identification and control based on modular neural networks (MNN) is proposed in this paper. The computational complexity of neural identification can be greatly reduced if the whole system is decomposed into several subsystems. This is obtained using a partitioning algorithm. Each local nonlinear model is associated with a nonlinear controller. These are also implemented by neural networks. The switching between the neural controllers is done by a dynamical switcher, also implemented by neural networks, that tracks the different operating points. The proposed multiple modelling and control strategy has been successfully tested on simulated laboratory scale liquid-level system.     

Network Connections That Evolve to Circumvent the Inverse Optics Problem

A fundamental problem in vision science is how useful perceptions and behaviors arise in the absence of information about the physical sources of retinal stimuli (the inverse optics problem). Psychophysical studies show that human observers contend with this problem by using the frequency of occurrence of stimulus patterns in cumulative experience to generate percepts. To begin to understand the neural mechanisms underlying this strategy, we examined the connectivity of simple neural networks evolved to respond according to the cumulative rank of stimulus luminance values. Evolved similarities with the connectivity of early level visual neurons suggests that biological visual circuitry uses the same mechanisms as a means of creating useful perceptions and behaviors without information about the real world.     

Potassium diffusive coupling in neural networks

Conventional neural networks are characterized by many neurons coupled together through synapses. The activity, synchronization, plasticity and excitability of the network are then controlled by its synaptic connectivity. Neurons are surrounded by an extracellular space whereby fluctuations in specific ionic concentration can modulate neuronal excitability. Extracellular concentrations of potassium ([K⁺]_o) can generate neuronal hyperexcitability. Yet, after many years of research, it is still unknown whether an elevation of potassium is the cause or the result of the generation, propagation and synchronization of epileptiform activity. An elevation of potassium in neural tissue can be characterized by dispersion (global elevation of potassium) and lateral diffusion (local spatial gradients). Both experimental and computational studies have shown that lateral diffusion is involved in the generation and the propagation of neural activity in diffusively coupled networks. Therefore, diffusion-based coupling by potassium can play an important role in neural networks and it is reviewed in four sections. Section 2 shows that potassium diffusion is responsible for the synchronization of activity across a mechanical cut in the tissue. A computer model of diffusive coupling shows that potassium diffusion can mediate communication between cells and generate abnormal and/or periodic activity in small (§3) and in large networks of cells (§4). Finally, in §5, a study of the role of extracellular potassium in the propagation of axonal signals shows that elevated potassium concentration can block the propagation of neural activity in axonal pathways. Taken together, these results indicate that potassium accumulation and diffusion can interfere with normal activity and generate abnormal activity in neural networks.     

A Neural Population Model Incorporating Dopaminergic Neurotransmission during Complex Voluntary Behaviors

Assessing brain activity during complex voluntary motor behaviors that require the recruitment of multiple neural sites is a field of active research. Our current knowledge is primarily based on human brain imaging studies that have clear limitations in terms of temporal and spatial resolution. We developed a physiologically informed non-linear multi-compartment stochastic neural model to simulate functional brain activity coupled with neurotransmitter release during complex voluntary behavior, such as speech production. Due to its state-dependent modulation of neural firing, dopaminergic neurotransmission plays a key role in the organization of functional brain circuits controlling speech and language and thus has been incorporated in our neural population model. A rigorous mathematical proof establishing existence and uniqueness of solutions to the proposed model as well as a computationally efficient strategy to numerically approximate these solutions are presented. Simulated brain activity during the resting state and sentence production was analyzed using functional network connectivity, and graph theoretical techniques were employed to highlight differences between the two conditions. We demonstrate that our model successfully reproduces characteristic changes seen in empirical data between the resting state and speech production, and dopaminergic neurotransmission evokes pronounced changes in modeled functional connectivity by acting on the underlying biological stochastic neural model. Specifically, model and data networks in both speech and rest conditions share task-specific network features: both the simulated and empirical functional connectivity networks show an increase in nodal influence and segregation in speech over the resting state. These commonalities confirm that dopamine is a key neuromodulator of the functional connectome of speech control. Based on reproducible characteristic aspects of empirical data, we suggest a number of extensions of the proposed methodology building upon the current model.     

Stochastic formulation for a partial neural circuit of <Emphasis Type="Italic">C. elegans</Emphasis>

We present a stochastic formulation for a partial neural circuit of Caenorhabditis elegans. This study is concerned with how to reduce the degree of freedom in a large neural circuit. In the presented formulation, neurons in the whole neural circuit are divided into two complementary groups. One is the neurons which are mainly associated with a certain behavior, and the other is the remaining neurons of C. elegans. In an ordinary study on a partial neural circuit, the influence of the latter (the remaining neurons) on the former (the associated neurons) is completely neglected. In the presented formulation, however, the influence is expressed by a stochastic variable. The structure of the ensemble for the stochastic variable is appropriately evaluated by the neural connectivity of C. elegans since the neural connectivity of C. elegans has been completely determined. In this way, the degree of freedom is effectively reduced. We apply the presented formulation to determine the synaptic signs in the touch sensitivity circuit of C. elegans. The synaptic signs are determined to satisfy the locomotory behaviors in C. elegans. We find that the influence of the remaining neurons on the touch sensitivity circuit is important to determine the synaptic signs.     

Flexible Memory Networks

Networks of neurons in some brain areas are flexible enough to encode new memories quickly. Using a standard firing rate model of recurrent networks, we develop a theory of flexible memory networks. Our main results characterize networks having the maximal number of flexible memory patterns, given a constraint graph on the network’s connectivity matrix. Modulo a mild topological condition, we find a close connection between maximally flexible networks and rank 1 matrices. The topological condition is H ₁(X;ℤ)=0, where X is the clique complex associated to the network’s constraint graph; this condition is generically satisfied for large random networks that are not overly sparse. In order to prove our main results, we develop some matrix-theoretic tools and present them in a self-contained section independent of the neuroscience context.     

Path formation in a robot swarm

We present two swarm intelligence control mechanisms used for distributed robot path formation. In the first, the robots form linear chains. We study three variants of robot chains, which vary in the degree of motion allowed to the chain structure. The second mechanism is called vectorfield. In this case, the robots form a pattern that globally indicates the direction towards a goal or home location. We test each controller on a task that consists in forming a path between two objects which an individual robot cannot perceive simultaneously. Our simulation experiments show promising results. All the controllers are able to form paths in complex obstacle environments and exhibit very good scalability, robustness, and fault tolerance characteristics. Additionally, we observe that chains perform better for small robot group sizes, while vectorfield performs better for large groups.     

Modular genetic control of innate behaviors

Many complex behaviors are genetically hardwired. Based on previous findings on genetic control of mating and other behaviors in invertebrate and mammalian systems, I suggest that genetic control of complex behaviors is modular: first, dedicated genes specify different behavioral patterns; secondly, separable genetic networks govern distinct behavioral components. I speculate that modular genetic encoding of complex behaviors may in part reflect modularity in brain development and function. Editor's suggested further reading in BioEssays From songs to synapses: Molecular mechanisms of birdsong memory Abstract     

Bursting Reverberation as a Multiscale Neuronal Network Process Driven by Synaptic Depression-Facilitation

Neuronal networks can generate complex patterns of activity that depend on membrane properties of individual neurons as well as on functional synapses. To decipher the impact of synaptic properties and connectivity on neuronal network behavior, we investigate the responses of neuronal ensembles from small (5–30 cells in a restricted sphere) and large (acute hippocampal slice) networks to single electrical stimulation: in both cases, a single stimulus generated a synchronous long-lasting bursting activity. While an initial spike triggered a reverberating network activity that lasted 2–5 seconds for small networks, we found here that it lasted only up to 300 milliseconds in slices. To explain this phenomena present at different scales, we generalize the depression-facilitation model and extracted the network time constants. The model predicts that the reverberation time has a bell shaped relation with the synaptic density, revealing that the bursting time cannot exceed a maximum value. Furthermore, before reaching its maximum, the reverberation time increases sub-linearly with the synaptic density of the network. We conclude that synaptic dynamics and connectivity shape the mean burst duration, a property present at various scales of the networks. Thus bursting reverberation is a property of sufficiently connected neural networks, and can be generated by collective depression and facilitation of underlying functional synapses.     

The relative efficiency of modular and non-modular networks of different size

Most biological networks are modular but previous work with small model networks has indicated that modularity does not necessarily lead to increased functional efficiency. Most biological networks are large, however, and here we examine the relative functional efficiency of modular and non-modular neural networks at a range of sizes. We conduct a detailed analysis of efficiency in networks of two size classes: ‘small’ and ‘large’, and a less detailed analysis across a range of network sizes. The former analysis reveals that while the modular network is less efficient than one of the two non-modular networks considered when networks are small, it is usually equally or more efficient than both non-modular networks when networks are large. The latter analysis shows that in networks of small to intermediate size, modular networks are much more efficient that non-modular networks of the same (low) connective density. If connective density must be kept low to reduce energy needs for example, this could promote modularity. We have shown how relative functionality/performance scales with network size, but the precise nature of evolutionary relationship between network size and prevalence of modularity will depend on the costs of connectivity.     

Do Brain Networks Evolve by Maximizing Their Information Flow Capacity?

We propose a working hypothesis supported by numerical simulations that brain networks evolve based on the principle of the maximization of their internal information flow capacity. We find that synchronous behavior and capacity of information flow of the evolved networks reproduce well the same behaviors observed in the brain dynamical networks of Caenorhabditis elegans and humans, networks of Hindmarsh-Rose neurons with graphs given by these brain networks. We make a strong case to verify our hypothesis by showing that the neural networks with the closest graph distance to the brain networks of Caenorhabditis elegans and humans are the Hindmarsh-Rose neural networks evolved with coupling strengths that maximize information flow capacity. Surprisingly, we find that global neural synchronization levels decrease during brain evolution, reflecting on an underlying global no Hebbian-like evolution process, which is driven by no Hebbian-like learning behaviors for some of the clusters during evolution, and Hebbian-like learning rules for clusters where neurons increase their synchronization.     

Dynamic large-scale connectivity of intrinsic cortical oscillations supports adaptive listening in challenging conditions

In multi-talker situations, individuals adapt behaviorally to this listening challenge mostly with ease, but how do brain neural networks shape this adaptation? We here establish a long-sought link between large-scale neural communications in electrophysiology and behavioral success in the control of attention in difficult listening situations. In an age-varying sample of N = 154 individuals, we find that connectivity between intrinsic neural oscillations extracted from source-reconstructed electroencephalography is regulated according to the listener’s goal during a challenging dual-talker task. These dynamics occur as spatially organized modulations in power-envelope correlations of alpha and low-beta neural oscillations during approximately 2-s intervals most critical for listening behavior relative to resting-state baseline. First, left frontoparietal low-beta connectivity (16 to 24 Hz) increased during anticipation and processing of a spatial-attention cue before speech presentation. Second, posterior alpha connectivity (7 to 11 Hz) decreased during comprehension of competing speech, particularly around target-word presentation. Connectivity dynamics of these networks were predictive of individual differences in the speed and accuracy of target-word identification, respectively, but proved unconfounded by changes in neural oscillatory activity strength. Successful adaptation to a listening challenge thus latches onto two distinct yet complementary neural systems: a beta-tuned frontoparietal network enabling the flexible adaptation to attentive listening state and an alpha-tuned posterior network supporting attention to speech.

This study investigates how intrinsic neural oscillations, acting in concert, tune into attentive listening. Using electroencephalography signals collected from people in a dual-talker listening task, the authors find that network connectivity of frontoparietal beta and posterior alpha oscillations is regulated according to the listener’s goal.