Robot control and the evolution of modular neurodynamics

Summary A modular approach to neural behavior control of autonomous robots is presented. It is based on the assumption that complex internal dynamics of recurrent neural networks can efficiently solve complex behavior tasks. For the development of appropriate neural control structures an evolutionary algorithm is introduced, which is able to generate neuromodules with specific functional properties, as well as the connectivity structure for a modular synthesis of such modules. This so called ENS ³-algorithm does not use genetic coding. It is primarily designed to develop size and connectivity structure of neuro-controllers. But at the same time it optimizes also parameters of individual networks like synaptic weights and bias terms. For demonstration, evolved networks for the control of miniature Khepera robots are presented. The aim is to develop robust controllers in the sense that neuro-controllers evolved in a simulator show comparably good behavior when loaded to a real robot acting in a physical environment. Discussed examples of such controllers generate obstacle avoidance and phototropic behaviors in non-trivial environments.     

Comparative strategies in the investigation of neural networks.

Comparative studies on nervous systems, though infrequently undertaken for the purpose of comparison, have yielded some important generalities about the formats of nervous networks, and about the cell biology of certain neural types. In the first category, it is clear that convergent evolutionary processes arrived at very similar networks to accomplish reciprocal and lateral inhibition, and load-compensation in "resistance reflexes." A newer general network format is described, command-derived inhibition, in which the central nervous elements controlling a rapid movement deliver presynaptic inhibition to the terminals of sensory neurons that carry reafferent excitation from the movement. It is argued that such circuits occur in several groups of animals, and that they include as a special class the efferent inhibitory neurons innervating acoustico-lateralis receptors in vertebrates. The properties of circuit elements that now seem to constitute useful generalizations include size principle (the inverse relationship between size and excitability in a variety of neurons), and the late differentiation of sensory neurons, failure to decussate, and their inability to mediate inhibition. Many other generalities have emerged, only to fall; one conclusion from such searches is that many supposedly "basic" properties of cell types or neural circuits are in fact not phylogenetically conservative, however much the physiologist may expect them to be.     

Homeostatic plasticity improves signal propagation in continuous-time recurrent neural networks

Continuous-time recurrent neural networks (CTRNNs) are potentially an excellent substrate for the generation of adaptive behaviour in artificial autonomous agents. However, node saturation effects in these networks can leave them insensitive to input and stop signals from propagating. Node saturation is related to the problems of hyper-excitation and quiescence in biological nervous systems, which are thought to be avoided through the existence of homeostatic plastic mechanisms. Analogous mechanisms are here implemented in a variety of CTRNN architectures and are shown to increase node sensitivity and improve signal propagation, with implications for robotics. These results lend support to the view that homeostatic plasticity may prevent quiescence and hyper-excitation in biological nervous systems.     

On the Relationships between Generative Encodings,Regularity, and Learning Abilities when Evolving Plastic Artificial Neural Networks

A major goal of bio-inspired artificial intelligence is to design artificial neural networks with abilities that resemble those of animal nervous systems. It is commonly believed that two keys for evolving nature-like artificial neural networks are (1) the developmental process that links genes to nervous systems, which enables the evolution of large, regular neural networks, and (2) synaptic plasticity, which allows neural networks to change during their lifetime. So far, these two topics have been mainly studied separately. The present paper shows that they are actually deeply connected. Using a simple operant conditioning task and a classic evolutionary algorithm, we compare three ways to encode plastic neural networks: a direct encoding, a developmental encoding inspired by computational neuroscience models, and a developmental encoding inspired by morphogen gradients (similar to HyperNEAT). Our results suggest that using a developmental encoding could improve the learning abilities of evolved, plastic neural networks. Complementary experiments reveal that this result is likely the consequence of the bias of developmental encodings towards regular structures: (1) in our experimental setup, encodings that tend to produce more regular networks yield networks with better general learning abilities; (2) whatever the encoding is, networks that are the more regular are statistically those that have the best learning abilities.     

Classical metapopulation dynamics and eco‐evolutionary feedbacks in dendritic networks

Eco‐evolutionary dynamics are now recognized to be highly relevant for population and community dynamics. However, the impact of evolutionary dynamics on spatial patterns, such as the occurrence of classical metapopulation dynamics, is less well appreciated. Here, we analyse the evolutionary consequences of spatial network connectivity and topology for dispersal strategies and quantify the eco‐evolutionary feedback in terms of altered classical metapopulation dynamics. We find that network properties, such as topology and connectivity, lead to predictable spatio‐temporal correlations in fitness expectations. These spatio‐temporally stable fitness patterns heavily impact evolutionarily stable dispersal strategies and lead to eco‐evolutionary feedbacks on landscape level metrics, such as the number of occupied patches, the number of extinctions and recolonizations as well as metapopulation extinction risk and genetic structure. Our model predicts that classical metapopulation dynamics are more likely to occur in dendritic networks, and especially in riverine systems, compared to other types of landscape configurations. As it remains debated whether classical metapopulation dynamics are likely to occur in nature at all, our work provides an important conceptual advance for understanding the occurrence of classical metapopulation dynamics which has implications for conservation and management of spatially structured populations.     

A neural network controller for the path tracking control of a hopping robot involving time delays

In this paper a hopping robot motion with offset mass is discussed. A mathematical model has been considered and an efficient single layered neural network has been developed to suit to the dynamics of the hopping robot, which ensures guaranteed tracking performance leading to the stability of the otherwise unstable system. The neural network takes advantage of the robot regressor dynamics that expresses the highly nonlinear robot dynamics in a linear form in terms of the known and unknown robot parameters. Time delays in the control mechanism play a vital role in the motion of hopping robots. The present work also enables us to estimate the maximum time delay admissible with out losing the guaranteed tracking performance. Further this neural network does not require offline training procedures. The salient features are highlighted by appropriate simulations.     

Neural morphogenesis, synaptic plasticity, and evolution

Summary Morphology plays an important role in the computational properties of neural systems, affecting both their functionality and the way in which this functionality is developed during life. In computer-based models of neural networks, artificial evolution is often used as a method to explore the space of suitable morphologies. In this paper we critically review the most common methods used to evolve neural morphologies and argue that a more effective, and possibly biologically plausible, method consists of genetically encoding rules of synaptic plasticity along with rules of neural morphogenesis. Some preliminary experiments with autonomous robots are described in order to show the feasibility and advantages of the approach.     

Enhancing evolvability with mutation buffering mediated through multiple weak interactions

The evolutionary adaptability of a system is dependent on three organizational properties, self-organizing dynamics that are hierarchically organized, component redundancy, and multiple weak interactions [Towards high evolvability dynamics, in: G. van de Vijver, S. Salthe, M. Delpos (Eds.), Evolutionary Systems, Kluwer Academic Publishers, Dordrecht, 1998, pp. 147-169]. This study reports on the use of the dual dynamics network model as an aid in understanding the role multiple weak interactions play in enhancing evolutionary adaptability. Dual dynamics networks are self-organizing systems that consist of simple components that change local state due to the coupled influences from connected components exerting strong discrete decision-making influences and from groups of components exerting multiple weak influences [J. Theor. Biol. 193 (1998) 287]. The dual dynamics model has been enhanced to support investigations of properties relevant to a system's capacity for evolvability, such as structure-function relationships, neutrality, adaptive tolerance, and evolutionary search performance.Three network types are investigated, each utilizing a different method of coupling strong and weak influences. The results demonstrate that the manner of coupling multiple weak interactions into the systems dynamics significantly affects the structure-function maps and the consequent evolvability characteristics. Specifically it is found that a form of coupling, denoted as linear modulation, enhances evolutionary adaptability. Linear modulation coupling requires that the weak interactions be integrated with strong interactions in a manner that implies a linear ordered relation between the possible state values of the components of the systems. When coupling functions that do not imply such an ordering of local state values are used, evolutionary adaptability is decreased.     

Application of chaotic dynamics in a recurrent neural network to control: hardware implementation into a novel autonomous roving robot

Originating from a viewpoint that complex/chaotic dynamics would play an important role in biological system including brains, chaotic dynamics introduced in a recurrent neural network was applied to control. The results of computer experiment was successfully implemented into a novel autonomous roving robot, which can only catch rough target information with uncertainty by a few sensors. It was employed to solve practical two-dimensional mazes using adaptive neural dynamics generated by the recurrent neural network in which four prototype simple motions are embedded. Adaptive switching of a system parameter in the neural network results in stationary motion or chaotic motion depending on dynamical situations. The results of hardware implementation and practical experiment using it show that, in given two-dimensional mazes, the robot can successfully avoid obstacles and reach the target. Therefore, we believe that chaotic dynamics has novel potential capability in controlling, and could be utilized to practical engineering application.     

Evolutionary changes of metabolic networks and their biosynthetic capacities

The metabolic networks of different species show a large variety in their structural design. In this work, the evolution of functional properties of metabolism in relation with metabolic network structure is investigated. The metabolism of ancestral species is inferred from the metabolism of contemporary species using a Bayesian network model for metabolism evolution. Subsequently, these networks are analysed with the recently developed method of network expansion. This method allows for a structural analysis of metabolic networks as well as a quantification of network functions in terms of their synthesising capacities when they are provided with certain external resources. The evolutionary dynamics of one particular network function: the metabolic expansion of glucose is investigated.     

Banknote recognition: investigating processing and cognition framework using competitive neural network

Humans are apt at recognizing patterns and discovering even abstract features which are sometimes embedded therein. Our ability to use the banknotes in circulation for business transactions lies in the effortlessness with which we can recognize the different banknote denominations after seeing them over a period of time. More significant is that we can usually recognize these banknote denominations irrespective of what parts of the banknotes are exposed to us visually. Furthermore, our recognition ability is largely unaffected even when these banknotes are partially occluded. In a similar analogy, the robustness of intelligent systems to perform the task of banknote recognition should not collapse under some minimum level of partial occlusion. Artificial neural networks are intelligent systems which from inception have taken many important cues related to structure and learning rules from the human nervous/cognition processing system. Likewise, it has been shown that advances in artificial neural network simulations can help us understand the human nervous/cognition system even furthermore. In this paper, we investigate three cognition hypothetical frameworks to vision-based recognition of banknote denominations using competitive neural networks. In order to make the task more challenging and stress-test the investigated hypotheses, we also consider the recognition of occluded banknotes. The implemented hypothetical systems are tasked to perform fast recognition of banknotes with up to 75 % occlusion. The investigated hypothetical systems are trained on Nigeria’s Naira banknotes and several experiments are performed to demonstrate the findings presented within this work.     

Distribution Coding in the Visual Pathway

Although a variety of types of spike interval histograms have been reported, little attention has been given to the spike interval distribution as a neural code and to how different distributions are transmitted through neural networks. In this paper we present experimental results showing spike interval histograms recorded from retinal ganglion cells of the cat. These results exhibit a clear correlation between spike interval distribution and stimulus condition at the retinal ganglion cell level. The averaged mean rates of the cells studied were nearly the same in light as in darkness whereas the spike interval histograms were much more regular in light than in darkness. We present theoretical models which illustrate how such a distribution coding at the retinal level could be “interpreted” or recorded at some higher level of the nervous system such as the lateral geniculate nucleus. Interpretation is an essential requirement of a neural code which has often been overlooked in modeling studies. Analytical expressions are derived describing the role of distribution coding in determining the transfer characteristics of a simple interaction model and of a lateral inhibition network. Our work suggests that distribution coding might be interpreted by simply interconnected neural networks such as relay cell networks, in general, and the primary thalamic sensory nuclei in particular.     

Introduction to the Symposium: What is Computational Neurobiology?

SYNOPSIS. Current advances in computational biology are derivedfrom two sets of ideas established in the work of Santiago Ramóny Cajal. One is the neuron doctrine, which holds that neuronsare the functional units of the nervous system, and has ledto detailed models of neuronal properties based on increasinginformation on the physical and chemical properties of neurons.The second is the idea of networks of neurons with specificpatterns of interconnections that has led to a variety of mathematicalmethods of analyzing such networks. Future work in computationalneurobiology promises to be a blend of these two modeling approaches.     

Learning Universal Computations with Spikes

Providing the neurobiological basis of information processing in higher animals, spiking neural networks must be able to learn a variety of complicated computations, including the generation of appropriate, possibly delayed reactions to inputs and the self-sustained generation of complex activity patterns, e.g. for locomotion. Many such computations require previous building of intrinsic world models. Here we show how spiking neural networks may solve these different tasks. Firstly, we derive constraints under which classes of spiking neural networks lend themselves to substrates of powerful general purpose computing. The networks contain dendritic or synaptic nonlinearities and have a constrained connectivity. We then combine such networks with learning rules for outputs or recurrent connections. We show that this allows to learn even difficult benchmark tasks such as the self-sustained generation of desired low-dimensional chaotic dynamics or memory-dependent computations. Furthermore, we show how spiking networks can build models of external world systems and use the acquired knowledge to control them.     

Virtual rigid body dynamics

An important direction in biological simulations is the development of methods that permit the study of larger systems and/or longer simulation time scales than is currently feasible by molecular dynamics. One such method designed with this objective in mind is stochastic boundary molecular dynamics (SBMD). SBMD was developed for the characterization of spatially localized processes in proteins, and has been shown to successfully reproduce structural and dynamical properties of these macromolecules, as compared to a molecular dynamics control simulation, when concerted or global motions are not present. The virtual rigid body dynamics method presented in this work extends the range of applicability of the SBMD method, by providing a framework to include these important long time scale conformational transitions. In this paper we describe the two-step implementation of the virtual rigid body model: first, the reduction of the full atomic representation to a reduced particle (virtual bond) model, and second, the propagation of the dynamics of flexibly connected rigid bodies containing virtual atom sites.     

Stimulus-induced bifurcations in discrete-time neural oscillators

Based on theoretical issues and neurobiological evidence, considerable interest has recently focused on dynamic computational elements in neural systems. Such elements respond to stimuli by altering their dynamical behavior rather than by changing a scalar output. In particular, neural oscillators capable of chaotic dynamics represent a potentially very rich substrate for complex spatiotemporal information processing. However, the response properties of such systems must be studied in detail before they can be used as computational elements in neural models. In this paper, we focus on the response of a very simple discrete-time neural oscillator model to a fixed input. We show that the oscillator responds to the stimulus through a fairly complex set of bifurcations, and shows critical switching between attractors. This information can be used to construct very sophisticated dynamic computational elements with well-understood response properties. Examples of such elements are presented in the paper. We end with a brief discussion of simple architectures for networks of dynamical elements, and the relevance of our results to neurobiological models. Received: 7 August 1997 / Accepted in revised form: 22 April 1998     

Inevitable evolutionary temporal elements in neural processing: a study based on evolutionary simulations

Recent studies have suggested that some neural computational mechanisms are based on the fine temporal structure of spiking activity. However, less effort has been devoted to investigating the evolutionary aspects of such mechanisms. In this paper we explore the issue of temporal neural computation from an evolutionary point of view, using a genetic simulation of the evolutionary development of neural systems. We evolve neural systems in an environment with selective pressure based on mate finding, and examine the temporal aspects of the evolved systems. In repeating evolutionary sessions, there was a significant increase during evolution in the mutual information between the evolved agent's temporal neural representation and the external environment. In ten different simulated evolutionary sessions, there was an increased effect of time-related neural ablations on the agents' fitness. These results suggest that in some fitness landscapes the emergence of temporal elements in neural computation is almost inevitable. Future research using similar evolutionary simulations may shed new light on various biological mechanisms.     

Phenotypic variability promotes diversity and stability in competitive communities

Intraspecific variation is at the core of evolutionary theory, and yet, from an ecological perspective, we have few robust expectations for how this variation should affect the dynamics of large communities. Here, by adapting an approach from evolutionary game theory, we show that the incorporation of phenotypic variability into competitive networks dramatically alters the dynamics across ecological timescales, stabilising the systems and buffering the communities against demographic perturbations. The beneficial effects of phenotypic variability are strongest when there are substantial differences among phenotypes and when phenotypes are inherited with moderately high fidelity; yet even low levels of variation lead to significant increases in diversity, stability, and robustness. By identifying a simple and ubiquitous stabilising force in competitive communities, this work contributes to our core understanding of how biological diversity is maintained in natural systems.     

How (not) to model autonomous behaviour

Autonomous systems are the result of self-sustaining processes of constitution of an identity under precarious circumstances. They may transit through different modes of dynamical engagement with their environment, from committed ongoing coping to open susceptibility to external demands. This paper discusses these two statements and presents examples of models of autonomous behaviour using methods in evolutionary robotics. A model of an agent capable of issuing self-instructions demonstrates the fragility of modelling autonomy as a function rather than as a property of a system's organization. An alternative model of behavioural preference based on homeostatic adaptation avoids this problem by establishing a mutual constraining between lower-level processes (neural dynamics and sensorimotor interaction) and higher-level metadynamics (experience-dependent, homeostatic triggering of local plasticity and re-organization). The results of these models are lessons about how strong autonomy should be approached: neither as a function, nor as a matter of external vs. internal determination.