Application of chaotic dynamics in a recurrent neural network to control: hardware implementation into a novel autonomous roving robot

Originating from a viewpoint that complex/chaotic dynamics would play an important role in biological system including brains, chaotic dynamics introduced in a recurrent neural network was applied to control. The results of computer experiment was successfully implemented into a novel autonomous roving robot, which can only catch rough target information with uncertainty by a few sensors. It was employed to solve practical two-dimensional mazes using adaptive neural dynamics generated by the recurrent neural network in which four prototype simple motions are embedded. Adaptive switching of a system parameter in the neural network results in stationary motion or chaotic motion depending on dynamical situations. The results of hardware implementation and practical experiment using it show that, in given two-dimensional mazes, the robot can successfully avoid obstacles and reach the target. Therefore, we believe that chaotic dynamics has novel potential capability in controlling, and could be utilized to practical engineering application.     

A biologically inspired neural net for trajectory formation and obstacle avoidance

In this paper we present a biologically inspired two-layered neural network for trajectory formation and obstacle avoidance. The two topographically ordered neural maps consist of analog neurons having continuous dynamics. The first layer, the sensory map, receives sensory information and builds up an activity pattern which contains the optimal solution (i.e. shortest path without collisions) for any given set of current position, target positions and obstacle positions. Targets and obstacles are allowed to move, in which case the activity pattern in the sensory map will change accordingly. The time evolution of the neural activity in the second layer, the motor map, results in a moving cluster of activity, which can be interpreted as a population vector. Through the feedforward connections between the two layers, input of the sensory map directs the movement of the cluster along the optimal path from the current position of the cluster to the target position. The smooth trajectory is the result of the intrinsic dynamics of the network only. No supervisor is required. The output of the motor map can be used for direct control of an autonomous system in a cluttered environment or for control of the actuators of a biological limb or robot manipulator. The system is able to reach a target even in the presence of an external perturbation. Computer simulations of a point robot and a multi-joint manipulator illustrate the theory.     

Human-like Manipulation Planning for Articulated Manipulator

A new method based on human-likeness assessment and optimization concept to solve the problem of human-like manipulation planning for articulated robot is proposed in this paper.This method intrinsically formulates the problem as a constrained optimization problem in robot configuration space.The robot configuration space is divided into different subregions by human likeness assessment.A widely used strategy,Rapid Upper Limb Assessment (RULA) in applied ergonomics,is adopted here to evaluate the human likeness of robot configuration.A task compatibility measurement of the robot velocity transmission ratio along a specified direction is used as the target function for the optimization problem.Simple illustrative examples of this method applied to a two Degree of Freedom (DOF) planar robot that resembles the upper limb of a human are presented.Further applications to a humanoid industrial robot SDA10D are also presented.The reasonable planning results for these applications assert the effectiveness of our method.     

A neural network controller for the path tracking control of a hopping robot involving time delays

In this paper a hopping robot motion with offset mass is discussed. A mathematical model has been considered and an efficient single layered neural network has been developed to suit to the dynamics of the hopping robot, which ensures guaranteed tracking performance leading to the stability of the otherwise unstable system. The neural network takes advantage of the robot regressor dynamics that expresses the highly nonlinear robot dynamics in a linear form in terms of the known and unknown robot parameters. Time delays in the control mechanism play a vital role in the motion of hopping robots. The present work also enables us to estimate the maximum time delay admissible with out losing the guaranteed tracking performance. Further this neural network does not require offline training procedures. The salient features are highlighted by appropriate simulations.     

Integrating neuromorphic action-oriented perceptual inputs to generate a navigation behaviour for a robot

We use neural networks with pointer map architectures to provide simple attentional processing in a robotic task. A pointer map comprises a map of neurons that encode a stimulus. Besides global feedback inhibition, the map receives feedback excitation via a small group of pointer neurons that encode the location of a salient stimulus on the map as a vectorial representation. The pointer neurons are able to apply selective processing to a particular region of the network. The robot uses these properties to manoeuver in relation to an attended object. We implemented a controller composed of two pointer maps, and a motor map. The first pointer map reports the direction of a salient obstacle in a one-dimensional map of distance derived from infrared sensors. The second pointer map reports the direction to potential obstacles in a two-dimensional edge-enhanced image derived from a forward looking CCD-camera. These outputs are applied to a motor map, where they bias the motor control signals issued to the robots wheels, according to navigational intentions.     

Efficient mapping of ligand migration channel networks in dynamic proteins

For many proteins such as myoglobin, the binding site lies in the interior, and there is no obvious route from the exterior to the binding site in the average structure. Although computer simulations for a limited number of proteins have found some transiently open channels, it is not clear if there exist more channels elsewhere or how the channels are regulated. A systematic approach that can map out the whole ligand migration channel network is lacking. Ligand migration in a dynamic protein resembles closely a well-studied problem in robotics, namely, the navigation of a mobile robot in a dynamic environment. In this work, we present a novel robotic motion planning inspired approach that can map the ligand migration channel network in a dynamic protein. The method combines an efficient spatial mapping of protein inner space with a temporal exploration of protein structural heterogeneity, which is represented by a structure ensemble. The spatial mapping of each conformation in the ensemble produces a partial map of protein inner cavities and their inter-connectivity. These maps are then merged to form a super map that contains all the channels that open dynamically. Results on the pathways in myoglobin for gaseous ligands demonstrate the efficiency of our approach in mapping the ligand migration channel networks. The results, obtained in a significantly less amount of time than trajectory-based approaches, are in agreement with previous simulation results. Additionally, the method clearly illustrates how and what conformational changes open or close a channel.     

Mantisbot is a robotic model of visually guided motion in the praying mantis

Insects use highly distributed nervous systems to process exteroception from head sensors, compare that information with state-based goals, and direct posture or locomotion toward those goals. To study how descending commands from brain centers produce coordinated, goal-directed motion in distributed nervous systems, we have constructed a conductance-based neural system for our robot MantisBot, a 29 degree-of-freedom, 13.3:1 scale praying mantis robot. Using the literature on mantis prey tracking and insect locomotion, we designed a hierarchical, distributed neural controller that establishes the goal, coordinates different joints, and executes prey-tracking motion. In our controller, brain networks perceive the location of prey and predict its future location, store this location in memory, and formulate descending commands for ballistic saccades like those seen in the animal. The descending commands are simple, indicating only 1) whether the robot should walk or stand still, and 2) the intended direction of motion. Each joint's controller uses the descending commands differently to alter sensory-motor interactions, changing the sensory pathways that coordinate the joints' central pattern generators into one cohesive motion. Experiments with one leg of MantisBot show that visual input produces simple descending commands that alter walking kinematics, change the walking direction in a predictable manner, enact reflex reversals when necessary, and can control both static posture and locomotion with the same network.     

Flexible structure multiple modeling using irregular self-organizing maps neural network

The MMSOM identification method, which had been presented by the authors, is improved to the multiple modeling by the irregular self-organizing map (MMISOM) using the irregular SOM (ISOM). Inputs to the neural networks are parameters of the instantaneous model computed adaptively at every instant. The neural network learns these models. The reference vectors of its output nodes are estimation of the parameters of the local models. At every instant, the model with closest output to the plant output is selected as the model of the plant. ISOM used in this paper is a graph of all the nodes and some of the weighted links between them to make a minimum spanning tree graph. It is shown in this paper that it is possible to add new models if the number of models is initially less than the appropriate one. The MMISOM shows more flexibility to cover the linear model space of the plant when the space is concave.     

Flexible structure multiple modeling using irregular self-organizing maps neural network

The MMSOM identification method, which had been presented by the authors, is improved to the multiple modeling by the irregular self-organizing map (MMISOM) using the irregular SOM (ISOM). Inputs to the neural networks are parameters of the instantaneous model computed adaptively at every instant. The neural network learns these models. The reference vectors of its output nodes are estimation of the parameters of the local models. At every instant, the model with closest output to the plant output is selected as the model of the plant. ISOM used in this paper is a graph of all the nodes and some of the weighted links between them to make a minimum spanning tree graph. It is shown in this paper that it is possible to add new models if the number of models is initially less than the appropriate one. The MMISOM shows more flexibility to cover the linear model space of the plant when the space is concave.     

Supervised Neural Q_learning based Motion Control for Bionic Underwater Robots

Bionic underwater robots have been a hot research area in recent years. The motion control methods for a kind of bionic underwater robot with two undulating fins are discussed in this paper. The equations of motion for the bionic underwater robot are described. To apply the reinforcement learning to the actual robot control, a Supervised Neural Q_learning (SNQL) algorithm is put forward. This algorithm is based on conventional Q_learning algorithm, but has three remarkable distinctions: (1) using a feedforward neural network to approximate the Q_function table; (2) adopting a learning sample database to speed up learning and improve the stability of learning system; (3) introducing a supervised control in the earlier stage of learning for safety and to speed up learning again. Experiments of swimming straightforward are carried out with SNQL algorithm. Results indicate that the SNQL algorithm is more effective than pure neural Q_learning or supervised control. It is a feasible approach to figure out the motion control for bionic underwater robots.     

Toward Fast and Efficient Mobility in Aquatic Environment: A Robot with Compliant Swimming Appendages Inspired by a Water Beetle

Water beetles are proficient drag-powered swimmers,with oar-like legs.Inspired by this mechamsm,here we propose a miniature robot,with mobility provided by a pair of legs with swimming appendages.The robot has optimized linkage structure to maximize the stroke angle,which is actuated by a single DC motor with a series of gears and a spring.A simplified swimming appendage model is proposed to calculate the deflection due to the applied drag force,and is compared with simulated data using COMSOL Multiphysics.Also,the swimming appendages are optimized by considering their locations on the legs using two fitness functions,and six different configurations are selected.We investigate the performance of the robot with various types of appendage using a high-speed camera,and motion capture cameras.The robot with the proposed configuration exhibits fast and efficient movement compared with other robots.In addition,the locomotion of the robot is analyzed by considering its dynamics,and compared with that of a water boatman (Corixidae).     

From sensors to spikes: evolving receptive fields to enhance sensorimotor information in a robot-arm

In biological systems, instead of actual encoders at different joints, proprioception signals are acquired through distributed receptive fields. In robotics, a single and accurate sensor output per link (encoder) is commonly used to track the position and the velocity. Interfacing bio-inspired control systems with spiking neural networks emulating the cerebellum with conventional robots is not a straight forward task. Therefore, it is necessary to adapt this one-dimensional measure (encoder output) into a multidimensional space (inputs for a spiking neural network) to connect, for instance, the spiking cerebellar architecture; i.e. a translation from an analog space into a distributed population coding in terms of spikes. This paper analyzes how evolved receptive fields (optimized towards information transmission) can efficiently generate a sensorimotor representation that facilitates its discrimination from other "sensorimotor states". This can be seen as an abstraction of the Cuneate Nucleus (CN) functionality in a robot-arm scenario. We model the CN as a spiking neuron population coding in time according to the response of mechanoreceptors during a multi-joint movement in a robot joint space. An encoding scheme that takes into account the relative spiking time of the signals propagating from peripheral nerve fibers to second-order somatosensory neurons is proposed. Due to the enormous number of possible encodings, we have applied an evolutionary algorithm to evolve the sensory receptive field representation from random to optimized encoding. Following the nature-inspired analogy, evolved configurations have shown to outperform simple hand-tuned configurations and other homogenized configurations based on the solution provided by the optimization engine (evolutionary algorithm). We have used artificial evolutionary engines as the optimization tool to circumvent nonlinearity responses in receptive fields.     

A space-time delay neural network for motion recognition and its application to lipreading

Motion recognition has received increasing attention in recent years owing to heightened demand for computer vision in many domains, including the surveillance system, multimodal human computer interface, and traffic control system. Most conventional approaches classify the motion recognition task into partial feature extraction and time-domain recognition subtasks. However, the information of motion resides in the space-time domain instead of the time domain or space domain independently, implying that fusing the feature extraction and classification in the space and time domains into a single framework is preferred. Based on this notion, this work presents a novel Space-Time Delay Neural Network (STDNN) capable of handling the space-time dynamic information for motion recognition. The STDNN is unified structure, in which the low-level spatiotemporal feature extraction and high-level space-time-domain recognition are fused. The proposed network possesses the spatiotemporal shift-invariant recognition ability that is inherited from the time delay neural network (TDNN) and space displacement neural network (SDNN), where TDNN and SDNN are good at temporal and spatial shift-invariant recognition, respectively. In contrast to multilayer perceptron (MLP), TDNN, and SDNN, STDNN is constructed by vector-type nodes and matrix-type links such that the spatiotemporal information can be accurately represented in a neural network. Also evaluated herein is the performance of the proposed STDNN via two experiments. The moving Arabic numerals (MAN) experiment simulates the object's free movement in the space-time domain on image sequences. According to these results, STDNN possesses a good generalization ability with respect to the spatiotemporal shift-invariant recognition. In the lipreading experiment, STDNN recognizes the lip motions based on the inputs of real image sequences. This observation confirms that STDNN yields a better performance than the existing TDNN-based system, particularly in terms of the generalization ability. In addition to the lipreading application, the STDNN can be applied to other problems since no domain-dependent knowledge is used in the experiment.     

From view cells and place cells to cognitive map learning: processing stages of the hippocampal system

 The goal of this paper is to propose a model of the hippocampal system that reconciles the presence of neurons that look like “place cells” with the implication of the hippocampus (Hs) in other cognitive tasks (e.g., complex conditioning acquisition and memory tasks). In the proposed model, “place cells” or “view cells” are learned in the perirhinal and entorhinal cortex. The role of the Hs is not fundamentally dedicated to navigation or map building, the Hs is used to learn, store, and predict transitions between multimodal states. This transition prediction mechanism could be important for novelty detection but, above all, it is crucial to merge planning and sensory–motor functions in a single and coherent system. A neural architecture embedding this model has been successfully tested on an autonomous robot, during navigation and planning in an open environment. Received: 28 June 1999 / Accepted in revised form: 26 April 2001     

Classification des stades de sommeil par des réseaux de neurones artificiels hiérarchiques

The goal of our work is to provide an automatic analysis and decision tool for sleep stages classification based on an artificial neural networks (ANN). The first difficulty lies in choosing the physiological signals representation and in particular the electroencephalogram (EEG). Once the representation adopted, the next step is to design the optimal neural network determined by a learning and validation process of data from a set of sleep records. We studied several configurations of conventional ANN giving results varying from 62 to 71 %, then we proposed a new hierarchical configuration, which gives a rate of 74 % correct classification for six stages. These results lead us to further explore this issue at the representation and design of ANNs to improve the performance of our tool.     

Learning to reach by reinforcement learning using a receptive field based function approximation approach with continuous actions

Reinforcement learning methods can be used in robotics applications especially for specific target-oriented problems, for example the reward-based recalibration of goal directed actions. To this end still relatively large and continuous state-action spaces need to be efficiently handled. The goal of this paper is, thus, to develop a novel, rather simple method which uses reinforcement learning with function approximation in conjunction with different reward-strategies for solving such problems. For the testing of our method, we use a four degree-of-freedom reaching problem in 3D-space simulated by a two-joint robot arm system with two DOF each. Function approximation is based on 4D, overlapping kernels (receptive fields) and the state-action space contains about 10,000 of these. Different types of reward structures are being compared, for example, reward-on- touching-only against reward-on-approach. Furthermore, forbidden joint configurations are punished. A continuous action space is used. In spite of a rather large number of states and the continuous action space these reward/punishment strategies allow the system to find a good solution usually within about 20 trials. The efficiency of our method demonstrated in this test scenario suggests that it might be possible to use it on a real robot for problems where mixed rewards can be defined in situations where other types of learning might be difficult. This work was supported by EU-Grant PACO-PLUS.     

Compact internal representation of dynamic situations: neural network implementing the causality principle

Animals for survival in complex, time-evolving environments can estimate in a “single parallel run” the fitness of different alternatives. Understanding of how the brain makes an effective compact internal representation (CIR) of such dynamic situations is a challenging problem. We propose an artificial neural network capable of creating CIRs of dynamic situations describing the behavior of a mobile agent in an environment with moving obstacles. The network exploits in a mental world model the principle of causality, which enables reduction of the time-dependent structure of real situations to compact static patterns. It is achieved through two concurrent processes. First, a wavefront representing the agent’s virtual present interacts with mobile and immobile obstacles forming static effective obstacles in the network space. The dynamics of the corresponding neurons in the virtual past is frozen. Then the diffusion-like process relaxes the remaining neurons to a stable steady state, i.e., a CIR is given by a single point in the multidimensional phase space. Such CIRs can be unfolded into real space for execution of motor actions, which allows a flexible task-dependent path planning in realistic time-evolving environments. Besides, the proposed network can also work as a part of “autonomous thinking”, i.e., some mental situations can be supplied for evaluation without direct motor execution. Finally we hypothesize the existence of a specific neuronal population responsible for detection of possible time-space coincidences of the animal and moving obstacles.     

A high-order graph generating self-organizing structure

A large class of neural network models have their units organized in a lattice with fixed topology or generate their topology during the learning process. These network models can be used as neighborhood preserving map of the input manifold, but such a structure is difficult to manage since these maps are graphs with a number of nodes that is just one or two orders of magnitude less than the number of input points (i.e., the complexity of the map is comparable with the complexity of the manifold) and some hierarchical algorithms were proposed in order to obtain a high-level abstraction of these structures. In this paper a general structure capable to extract high order information from the graph generated by a large class of self-organizing networks is presented. This algorithm will allow to build a two layers hierarchical structure starting from the results obtained by using the suitable neural network for the distribution of the input data. Moreover the proposed algorithm is also capable to build a topology preserving map if it is trained using a graph that is also a topology preserving map.     

Purposive behavior and cognitive mapping: a neural network model

This study presents a real-time, biologically plausible neural network approach to purposive behavior and cognitive mapping. The system is composed of (a) an action system, consisting of a goal-seeking neural mechanism controlled by a motivational system; and (b) a cognitive system, involving a neural cognitive map. The goal-seeking mechanism displays exploratory behavior until either (a) the goal is found or (b) an adequate prediction of the goal is generated. The cognitive map built by the network is a top logical map, i.e., it represents only the adjacency, but not distances or directions, between places. The network has recurrent and non-recurrent properties that allow the reading of the cognitive map without modifying it. Two types of predictions are introduced: fast-time and real-time predictions. Fast-time predictions are produced in advance of what occurs in real time, when the information stored in the cognitive map is used to predict the remote future. Real-time predictions are generated simultaneously with the occurrence of environmental events, when the information stored in the cognitive map is being updated. Computer simulations show that the network successfully describes latent learning and detour behavior in rats. In addition, simulations demonstrate that the network can be applied to problem-solving paradigms such as the Tower of Hanoi puzzle.     

Development of a Biomimetic Quadruped Robot

This paper presents the design and prototype of a small quadruped robot whose walking motion is realized by two piezocomposite actuators. In the design, biomimetic ideas are employed to obtain the agility of motions and sustainability of a heavy load. The design of the robot legs is inspired by the leg configuration of insects, two joints （hip and knee） of the leg enable two basic motions, lifting and stepping. The robot frame is designed to have a slope relative to the horizontal plane, which makes the robot move forward. In addition, the bounding locomotion of quadruped animals is implemented in the robot. Experiments show that the robot can carry an additional load of about 100 g and run with a fairly high velocity. The quadruped prototype can be an important step towards the goal of building an autonomous mobile robot actuated by piezocomposite actuators.