Evolving cognitive-behavioural dependencies in situated agents for behavioural robustness

This article investigates the emergence of robust behaviour in agents with dynamically limited controllers (monostable agents), and compares their performance to less limited ones (bistable agents). 'Dynamically limited' here refers to a reduced quantity of steady states that an agent controller exhibits when it does not receive stimulus from the environment. Agents are evolved for categorical perception, a minimal cognitive task, and must correlate approaching or avoiding movements based on (two) different types of objects. Results indicate a significant tendency to better behavioural robustness by monostable in contrast to bistable agents in the presence of sensorimotor, mutational, and structural perturbations. Discussions here focus on a further dependence to coupled dynamics by the former agents to explain such a tendency.     

A multisensory integration model of human stance control

A model is presented to study and quantify the contribution of all available sensory information to human standing based on optimal estimation theory. In the model, delayed sensory information is integrated in such a way that a best estimate of body orientation is obtained. The model approach agrees with the present theory of the goal of human balance control. The model is not based on purely inverted pendulum body dynamics, but rather on a three-link segment model of a standing human on a movable support base. In addition, the model is non-linear and explicitly addresses the problem of multisensory integration and neural time delays. A predictive element is included in the controller to compensate for time delays, necessary to maintain erect body orientation. Model results of sensory perturbations on total body sway closely resemble experimental results. Despite internal and external perturbations, the controller is able to stabilise the model of an inherently unstable standing human with neural time delays of 100 ms. It is concluded, that the model is capable of studying and quantifying multisensory integration in human stance control. We aim to apply the model in (1) the design and development of prostheses and orthoses and (2) the diagnosis of neurological balance disorders. Received: 25 August 1997 / Accepted in revised form: 8 December 1998     

Synchronisation effects on the behavioural performance and information dynamics of a simulated minimally cognitive robotic agent

Oscillatory activity is ubiquitous in nervous systems, with solid evidence that synchronisation mechanisms underpin cognitive processes. Nevertheless, its informational content and relationship with behaviour are still to be fully understood. In addition, cognitive systems cannot be properly appreciated without taking into account brain-body- environment interactions. In this paper, we developed a model based on the Kuramoto Model of coupled phase oscillators to explore the role of neural synchronisation in the performance of a simulated robotic agent in two different minimally cognitive tasks. We show that there is a statistically significant difference in performance and evolvability depending on the synchronisation regime of the network. In both tasks, a combination of information flow and dynamical analyses show that networks with a definite, but not too strong, propensity for synchronisation are more able to reconfigure, to organise themselves functionally and to adapt to different behavioural conditions. The results highlight the asymmetry of information flow and its behavioural correspondence. Importantly, it also shows that neural synchronisation dynamics, when suitably flexible and reconfigurable, can generate minimally cognitive embodied behaviour.     

A hierarchical neural-network model for control and learning of voluntary movement

In order to control voluntary movements, the central nervous system (CNS) must solve the following three computational problems at different levels: the determination of a desired trajectory in the visual coordinates, the transformation of its coordinates to the body coordinates and the generation of motor command. Based on physiological knowledge and previous models, we propose a hierarchical neural network model which accounts for the generation of motor command. In our model the association cortex provides the motor cortex with the desired trajectory in the body coordinates, where the motor command is then calculated by means of long-loop sensory feedback. Within the spinocerebellum — magnocellular red nucleus system, an internal neural model of the dynamics of the musculoskeletal system is acquired with practice, because of the heterosynaptic plasticity, while monitoring the motor command and the results of movement. Internal feedback control with this dynamical model updates the motor command by predicting a possible error of movement. Within the cerebrocerebellum — parvocellular red nucleus system, an internal neural model of the inverse-dynamics of the musculo-skeletal system is acquired while monitoring the desired trajectory and the motor command. The inverse-dynamics model substitutes for other brain regions in the complex computation of the motor command. The dynamics and the inverse-dynamics models are realized by a parallel distributed neural network, which comprises many sub-systems computing various nonlinear transformations of input signals and a neuron with heterosynaptic plasticity (that is, changes of synaptic weights are assumed proportional to a product of two kinds of synaptic inputs). Control and learning performance of the model was investigated by computer simulation, in which a robotic manipulator was used as a controlled system, with the following results: (1) Both the dynamics and the inverse-dynamics models were acquired during control of movements. (2) As motor learning proceeded, the inverse-dynamics model gradually took the place of external feedback as the main controller. Concomitantly, overall control performance became much better. (3) Once the neural network model learned to control some movement, it could control quite different and faster movements. (4) The neural netowrk model worked well even when only very limited information about the fundamental dynamical structure of the controlled system was available. Consequently, the model not only accounts for the learning and control capability of the CNS, but also provides a promising parallel-distributed control scheme for a large-scale complex object whose dynamics are only partially known.     

Candidate neural locus for sex differences in reproductive decisions

Sexual selection and signal detection theories predict that females should be selective in their responses to mating signals in mate choice, while the response of males to signals in male competition should be less selective. The neural processes underlying this behavioural sex difference remain obscure. Differences in behavioural selectivity could result from differences in how sensitive sensory systems are to mating signals, distinct thresholds in motor areas regulating behaviour, or sex differences in selectivity at a gateway relaying sensory information to motor systems. We tested these hypotheses in frogs using the expression of egr-1 to quantify the neural responses of each sex to mating signals. We found that egr-1 expression in a midbrain auditory region was elevated in males in response to both conspecific and heterospecific calls, whereas in females, egr-1 induction occurred only in response to conspecific signals. This differential neural selectivity mirrored the sex differences in behavioural responsiveness to these stimuli. By contrast, egr-1 expression in lower brainstem auditory centres was not different in males and females. Our results support a model in which sex differences in behavioural selectivity arise from sex differences in the neural selectivity in midbrain areas relaying sensory information to the forebrain.     

Visual influences on auditory spatial learning

The visual and auditory systems frequently work together to facilitate the identification and localization of objects and events in the external world. Experience plays a critical role in establishing and maintaining congruent visual-auditory associations, so that the different sensory cues associated with targets that can be both seen and heard are synthesized appropriately. For stimulus location, visual information is normally more accurate and reliable and provides a reference for calibrating the perception of auditory space. During development, vision plays a key role in aligning neural representations of space in the brain, as revealed by the dramatic changes produced in auditory responses when visual inputs are altered, and is used throughout life to resolve short-term spatial conflicts between these modalities. However, accurate, and even supra-normal, auditory localization abilities can be achieved in the absence of vision, and the capacity of the mature brain to relearn to localize sound in the presence of substantially altered auditory spatial cues does not require visuomotor feedback. Thus, while vision is normally used to coordinate information across the senses, the neural circuits responsible for spatial hearing can be recalibrated in a vision-independent fashion. Nevertheless, early multisensory experience appears to be crucial for the emergence of an ability to match signals from different sensory modalities and therefore for the outcome of audiovisual-based rehabilitation of deaf patients in whom hearing has been restored by cochlear implantation.     

Sensory feedback in CNN-based central pattern generators

Central Pattern Generators (CPGs) are a suitable paradigm to solve the problem of locomotion control in walking robots. CPGs are able to generate feed-forward signals to achieve a proper coordination among the robot legs. In literature they are often modelled as networks of coupled nonlinear systems. However the topic of feedback in these systems is rarely addressed. On the other hand feedback is essential for locomotion. In this paper the CPG for a hexapod robot is implemented through Cellular Neural Networks (CNNs). Feedback is included in the CPG controller by exploiting the dynamic properties of the CPG motor-neurons, such as synchronization issue and local bifurcations. These universal paradigms provide the essential issues to include sensory feedback in CPG architectures based on coupled nonlinear systems. Experiments on a dynamic model of a hexapod robot are presented to validate the approach introduced.     

Relating reflex gain modulation in posture control to underlying neural network properties using a neuromusculoskeletal model

During posture control, reflexive feedback allows humans to efficiently compensate for unpredictable mechanical disturbances. Although reflexes are involuntary, humans can adapt their reflexive settings to the characteristics of the disturbances. Reflex modulation is commonly studied by determining reflex gains: a set of parameters that quantify the contributions of Ia, Ib and II afferents to mechanical joint behavior. Many mechanisms, like presynaptic inhibition and fusimotor drive, can account for reflex gain modulations. The goal of this study was to investigate the effects of underlying neural and sensory mechanisms on mechanical joint behavior. A neuromusculoskeletal model was built, in which a pair of muscles actuated a limb, while being controlled by a model of 2,298 spiking neurons in six pairs of spinal populations. Identical to experiments, the endpoint of the limb was disturbed with force perturbations. System identification was used to quantify the control behavior with reflex gains. A sensitivity analysis was then performed on the neuromusculoskeletal model, determining the influence of the neural, sensory and synaptic parameters on the joint dynamics. The results showed that the lumped reflex gains positively correlate to their most direct neural substrates: the velocity gain with Ia afferent velocity feedback, the positional gain with muscle stretch over II afferents and the force feedback gain with Ib afferent feedback. However, position feedback and force feedback gains show strong interactions with other neural and sensory properties. These results give important insights in the effects of neural properties on joint dynamics and in the identifiability of reflex gains in experiments.     

Shaping the dynamics of a bidirectional neural interface

Progress in decoding neural signals has enabled the development of interfaces that translate cortical brain activities into commands for operating robotic arms and other devices. The electrical stimulation of sensory areas provides a means to create artificial sensory information about the state of a device. Taken together, neural activity recording and microstimulation techniques allow us to embed a portion of the central nervous system within a closed-loop system, whose behavior emerges from the combined dynamical properties of its neural and artificial components. In this study we asked if it is possible to concurrently regulate this bidirectional brain-machine interaction so as to shape a desired dynamical behavior of the combined system. To this end, we followed a well-known biological pathway. In vertebrates, the communications between brain and limb mechanics are mediated by the spinal cord, which combines brain instructions with sensory information and organizes coordinated patterns of muscle forces driving the limbs along dynamically stable trajectories. We report the creation and testing of the first neural interface that emulates this sensory-motor interaction. The interface organizes a bidirectional communication between sensory and motor areas of the brain of anaesthetized rats and an external dynamical object with programmable properties. The system includes (a) a motor interface decoding signals from a motor cortical area, and (b) a sensory interface encoding the state of the external object into electrical stimuli to a somatosensory area. The interactions between brain activities and the state of the external object generate a family of trajectories converging upon a selected equilibrium point from arbitrary starting locations. Thus, the bidirectional interface establishes the possibility to specify not only a particular movement trajectory but an entire family of motions, which includes the prescribed reactions to unexpected perturbations.     

Non-linear stimulus-response behavior of the human stance control system is predicted by optimization of a system with sensory and motor noise

We developed a theory of human stance control that predicted (1) how subjects re-weight their utilization of proprioceptive and graviceptive orientation information in experiments where eyes closed stance was perturbed by surface-tilt stimuli with different amplitudes, (2) the experimentally observed increase in body sway variability (i.e. the “remnant” body sway that could not be attributed to the stimulus) with increasing surface-tilt amplitude, (3) neural controller feedback gains that determine the amount of corrective torque generated in relation to sensory cues signaling body orientation, and (4) the magnitude and structure of spontaneous body sway. Responses to surface-tilt perturbations with different amplitudes were interpreted using a feedback control model to determine control parameters and changes in these parameters with stimulus amplitude. Different combinations of internal sensory and/or motor noise sources were added to the model to identify the properties of noise sources that were able to account for the experimental remnant sway characteristics. Various behavioral criteria were investigated to determine if optimization of these criteria could predict the identified model parameters and amplitude-dependent parameter changes. Robust findings were that remnant sway characteristics were best predicted by models that included both sensory and motor noise, the graviceptive noise magnitude was about ten times larger than the proprioceptive noise, and noise sources with signal-dependent properties provided better explanations of remnant sway. Overall results indicate that humans dynamically weight sensory system contributions to stance control and tune their corrective responses to minimize the energetic effects of sensory noise and external stimuli.     

From network-to-antibody robustness in a bio-inspired immune system

Behavioural robustness at antibody and immune network level is discussed. The robustness of the immune response that drives an autonomous mobile robot is examined with two computational experiments in the autonomous mobile robots trajectory generation context in unknown environments. The immune response is met based on the immune network metaphor for different low-level behaviours coordination. These behaviours are activated when a robot sense the appropriate conditions in the environment in relation to the network current state. Results are obtained over a case study in computer simulation as well as in laboratory experiments with a Khepera II microrobot. In this work, we develop a set of tests where such an immune response is externally perturbed at network or low-level behavioural modules to analyse the robust capacity of the system to unexpected perturbations. Emergence of robust behaviour and high-level immune response relates to the coupling between behavioural modules that are selectively engaged with the environment based on immune response. Experimental evidence leads discussions on a dynamical systems perspective of behavioural robustness in artificial immune systems that goes beyond the isolated immune network response.     

Is Empiricism Empirically False? Lessons from Early Nervous Systems

Recent work on skin-brain thesis (de Wiljes et al. 2015; Keijzer 2015; Keijzer et al. 2013) suggests the possibility of empirical evidence that empiricism is false. It implies that early animals need no traditional sensory receptors to be engaged in cognitive activity. The neural structure required to coordinate extensive sheets of contractile tissue for motility provides the starting point for a new multicellular organized form of sensing. Moving a body by muscle contraction provides the basis for a multicellular organization that is sensitive to external surface structure at the scale of the animal body. In other words, the nervous system first evolved for action, not for receiving sensory input. Thus, sensory input is not required for minimal cognition; only action is. The whole body of an organism, in particular its highly specific animal sensorimotor organization, reflects the bodily and environmental spatiotemporal structure. The skin-brain thesis suggests that, in contrast to empiricist claims that cognition is constituted by sensory systems, cognition may be also constituted by action-oriented feedback mechanisms. Instead of positing the reflex arc as the elementary building block of nervous systems, it proposes that endogenous motor activity is crucial for cognitive processes. In the paper, I discuss the issue whether the skin-brain thesis and its supporting evidence can be really used to overthrow the main tenet of empiricism empirically, by pointing out to cognizing agents that fail to have any sensory apparatus.     

A model of neuro-musculo-skeletal system for human locomotion under position constraint condition

The human locomotion was studied on the basis of the interaction of the musculo-skeletal system, the neural system and the environment. A mathematical model of human locomotion under position constraint condition was established. Besides the neural rhythm generator, the posture controller and the sensory system, the environment feedback controller and the stability controller were taken into account in the model. The environment feedback controller was proposed for two purposes, obstacle avoidance and target position control of the swing foot. The stability controller was proposed to imitate the self-balancing ability of a human body and improve the stability of the model. In the stability controller, the ankle torque was used to control the velocity of the body gravity center. A prediction control algorithm was applied to calculate the torque magnitude of the stability controller. As an example, human stairs climbing movement was simulated and the results were given. The simulation result proved that the mathematical modeling of the task was successful.     

Biases in signal evolution: learning makes a difference

It is now well established that signal receivers have a key role in the evolution of animal communication: the suite of sensory and cognitive processes by which animals perceive and learn about their environment can have a significant impact on signal design. A crucial property of these information-processing mechanisms is the emergence of 'receiver bias' in the behavioural responses to signals. Whereas most research has focussed on receiver biases in the sensory system, more recent studies show that biases can also arise from learning about signals. Here, we highlight how learning-based biases can arise, and how these differ from biases emerging from sensory systems in their impact on signal evolution.     

What we see is how we are: new paradigms in visual research

As most sensory modalities, the visual system needs to deal with very fast changes in the environment. Instead of processing all sensory stimuli, the brain is able to construct a perceptual experience by combining selected sensory input with an ongoing internal activity. Thus, the study of visual perception needs to be approached by examining not only the physical properties of stimuli, but also the brain's ongoing dynamical states onto which these perturbations are imposed. At least three different models account for this internal dynamics. One model is based on cardinal cells where the activity of few cells by itself constitutes the neuronal correlate of perception, while a second model is based on a population coding that states that the neuronal correlate of perception requires distributed activity throughout many areas of the brain. A third proposition, known as the temporal correlation hypothesis states that the distributed neuronal populations that correlate with perception, are also defined by synchronization of the activity on a millisecond time scale. This would serve to encode contextual information by defining relations between the features of visual objects. If temporal properties of neural activity are important to establish the neural mechanisms of perception, then the study of appropriate dynamical stimuli should be instrumental to determine how these systems operate. The use of natural stimuli and natural behaviors such as free viewing, which features fast changes of internal brain states as seen by motor markers, is proposed as a new experimental paradigm to study visual perception.     

A robotic device for understanding neuromechanical interactions during standing balance control

Postural stability in standing balance results from the mechanics of body dynamics as well as active neural feedback control processes. Even when an animal or human has multiple legs on the ground, active neural regulation of balance is required. When the postural configuration, or stance, changes, such as when the feet are placed further apart, the mechanical stability of the organism changes, but the degree to which this alters the demands on neural feedback control for postural stability is unknown. We developed a robotic system that mimics the neuromechanical postural control system of a cat in response to lateral perturbations. This simple robotic system allows us to study the interactions between various parameters that contribute to postural stability and cannot be independently varied in biological systems. The robot is a 'planar', two-legged device that maintains compliant balance control in a variety of stance widths when subject to perturbations of the support surface, and in this sense reveals principles of lateral balance control that are also applicable to bipeds. Here we demonstrate that independent variations in either stance width or delayed neural feedback gains can have profound and often surprisingly detrimental effects on the postural stability of the system. Moreover, we show through experimentation and analysis that changing stance width alters fundamental mechanical relationships important in standing balance control and requires a coordinated adjustment of delayed feedback control to maintain postural stability.     

Boolean dynamics of biological networks with multiple coupled feedback loops

Boolean networks have been frequently used to study the dynamics of biological networks. In particular, there have been various studies showing that the network connectivity and the update rule of logical functions affect the dynamics of Boolean networks. There has been, however, relatively little attention paid to the dynamical role of a feedback loop, which is a circular chain of interactions between Boolean variables. We note that such feedback loops are ubiquitously found in various biological systems as multiple coupled structures and they are often the primary cause of complex dynamics. In this article, we investigate the relationship between the multiple coupled feedback loops and the dynamics of Boolean networks. We show that networks have a larger proportion of basins corresponding to fixed-point attractors as they have more coupled positive feedback loops, and a larger proportion of basins for limit-cycle attractors as they have more coupled negative feedback loops.     

Driving reservoir models with oscillations: a solution to the extreme structural sensitivity of chaotic networks

A large body of experimental and theoretical work on neural coding suggests that the information stored in brain circuits is represented by time-varying patterns of neural activity. Reservoir computing, where the activity of a recurrently connected pool of neurons is read by one or more units that provide an output response, successfully exploits this type of neural activity. However, the question of system robustness to small structural perturbations, such as failing neurons and synapses, has been largely overlooked. This contrasts with well-studied dynamical perturbations that lead to divergent network activity in the presence of chaos, as is the case for many reservoir networks. Here, we distinguish between two types of structural network perturbations, namely local (e.g., individual synaptic or neuronal failure) and global (e.g., network-wide fluctuations). Surprisingly, we show that while global perturbations have a limited impact on the ability of reservoir models to perform various tasks, local perturbations can produce drastic effects. To address this limitation, we introduce a new architecture where the reservoir is driven by a layer of oscillators that generate stable and repeatable trajectories. This model outperforms previous implementations while being resistant to relatively large local and global perturbations. This finding has implications for the design of reservoir models that capture the capacity of brain circuits to perform cognitively and behaviorally relevant tasks while remaining robust to various forms of perturbations. Further, our work proposes a novel role for neuronal oscillations found in cortical circuits, where they may serve as a collection of inputs from which a network can robustly generate complex dynamics and implement rich computations.     

Behaviour-based modelling of hexapod locomotion: linking biology and technical application

Walking in insects and most six-legged robots requires simultaneous control of up to 18 joints. Moreover, the number of joints that are mechanically coupled via body and ground varies from one moment to the next, and external conditions such as friction, compliance and slope of the substrate are often unpredictable. Thus, walking behaviour requires adaptive, context-dependent control of many degrees of freedom. As a consequence, modelling legged locomotion addresses many aspects of any motor behaviour in general. Based on results from behavioural experiments on arthropods, we describe a kinematic model of hexapod walking: the distributed artificial neural network controller walknet. Conceptually, the model addresses three basic problems in legged locomotion. (I) First, coordination of several legs requires coupling between the step cycles of adjacent legs, optimising synergistic propulsion, but ensuring stability through flexible adjustment to external disturbances. A set of behaviourally derived leg coordination rules can account for decentralised generation of different gaits, and allows stable walking of the insect model as well as of a number of legged robots. (II) Second, a wide range of different leg movements must be possible, e.g. to search for foothold, grasp for objects or groom the body surface. We present a simple neural network controller that can simulate targeted swing trajectories, obstacle avoidance reflexes and cyclic searching-movements. (III) Third, control of mechanically coupled joints of the legs in stance is achieved by exploiting the physical interactions between body, legs and substrate. A local positive displacement feedback, acting on individual leg joints, transforms passive displacement of a joint into active movement, generating synergistic assistance reflexes in all mechanically coupled joints.