Unsupervised learning and temporal context to recall complex robot trajectories

An unsupervised neural network is proposed to learn and recall complex robot trajectories. Two cases are considered: (i) A single trajectory in which a particular arm configuration (state) may occur more than once, and (ii) trajectories sharing states with each other. Ambiguities occur in both cases during recall of such trajectories. The proposed model consists of two groups of synaptic weights trained by competitive and Hebbian learning laws. They are responsible for encoding spatial and temporal features of the input sequences, respectively. Three mechanisms allow the network to deal with repeated or shared states: local and global context units, neurons disabled from learning, and redundancy. The network reproduces the current and the next state of the learned sequences and is able to resolve ambiguities. The model was simulated over various sets of robot trajectories in order to evaluate learning and recall, trajectory sampling effects and robustness.     

Unsupervised learning and recall of temporal sequences: an application to robotics

This paper describes an unsupervised neural network model for learning and recall of temporal patterns. The model comprises two groups of synaptic weights, named competitive feedforward and Hebbian feedback, which are responsible for encoding the static and temporal features of the sequence respectively. Three additional mechanisms allow the network to deal with complex sequences: context units, a neuron commitment equation, and redundancy in the representation of sequence states. The proposed network encodes a set of robot trajectories which may contain states in common, and retrieves them accurately in the correct order. Further tests evaluate the fault-tolerance and noise sensitivity of the proposed model.     

Full-state tracking control of a mobile robot using neural networks

In this paper a nonholonomic mobile robot with completely unknown dynamics is discussed. A mathematical model has been considered and an efficient neural network is developed, which ensures guaranteed tracking performance leading to stability of the system. The neural network assumes a single layer structure, by taking advantage of the robot regressor dynamics that expresses the highly nonlinear robot dynamics in a linear form in terms of the known and unknown robot dynamic parameters. No assumptions relating to the boundedness is placed on the unmodeled disturbances. It is capable of generating real-time smooth and continuous velocity control signals that drive the mobile robot to follow the desired trajectories. The proposed approach resolves speed jump problem existing in some previous tracking controllers. Further, this neural network does not require offline training procedures. Lyapunov theory has been used to prove system stability. The practicality and effectiveness of the proposed tracking controller are demonstrated by simulation and comparison results.     

Interpretable machine learning for high-dimensional trajectories of aging health

We have built a computational model for individual aging trajectories of health and survival, which contains physical, functional, and biological variables, and is conditioned on demographic, lifestyle, and medical background information. We combine techniques of modern machine learning with an interpretable interaction network, where health variables are coupled by explicit pair-wise interactions within a stochastic dynamical system. Our dynamic joint interpretable network (DJIN) model is scalable to large longitudinal data sets, is predictive of individual high-dimensional health trajectories and survival from baseline health states, and infers an interpretable network of directed interactions between the health variables. The network identifies plausible physiological connections between health variables as well as clusters of strongly connected health variables. We use English Longitudinal Study of Aging (ELSA) data to train our model and show that it performs better than multiple dedicated linear models for health outcomes and survival. We compare our model with flexible lower-dimensional latent-space models to explore the dimensionality required to accurately model aging health outcomes. Our DJIN model can be used to generate synthetic individuals that age realistically, to impute missing data, and to simulate future aging outcomes given arbitrary initial health states.     

A Switching Controller System for a Wheeled Mobile Robot

A wheeled mobile mechanism with a passive and/or active linkage mechanism for rough terrain environment is developed and evaluated. The wheeled mobile mechanism which has high mobility in rough terrain needs sophisticated system to adapt various environments.We focus on the development of a switching controller system for wheeled mobile robots in rough terrain. This system consists of two sub-systems: an environment recognition system using link angles and an adaptive control system. In the environment recognition system, we introduce a Self-Organizing Map (SOM) for clustering link angles. In the adaptive controllers, we introduce neural networks to calculate the inverse model of the wheeled mobile robot.The environment recognition system can recognize the environment in which the robot travels, and the adjustable controllers are tuned by experimental results for each environment. The dual sub-system switching controller system is experimentally evaluated. The system recognizes its environment and adapts by switching the adjustable controllers. This system demonstrates superior performance to a well-tuned single PID controller.     

A neurodynamic account of spontaneous behaviour

The current article suggests that deterministic chaos self-organized in cortical dynamics could be responsible for the generation of spontaneous action sequences. Recently, various psychological observations have suggested that humans and primates can learn to extract statistical structures hidden in perceptual sequences experienced during active environmental interactions. Although it has been suggested that such statistical structures involve chunking or compositional primitives, their neuronal implementations in brains have not yet been clarified. Therefore, to reconstruct the phenomena, synthetic neuro-robotics experiments were conducted by using a neural network model, which is characterized by a generative model with intentional states and its multiple timescales dynamics. The experimental results showed that the robot successfully learned to imitate tutored behavioral sequence patterns by extracting the underlying transition probability among primitive actions. An analysis revealed that a set of primitive action patterns was embedded in the fast dynamics part, and the chaotic dynamics of spontaneously sequencing these action primitive patterns was structured in the slow dynamics part, provided that the timescale was adequately set for each part. It was also shown that self-organization of this type of functional hierarchy ensured robust action generation by the robot in its interactions with a noisy environment. This article discusses the correspondence of the synthetic experiments with the known hierarchy of the prefrontal cortex, the supplementary motor area, and the primary motor cortex for action generation. We speculate that deterministic dynamical structures organized in the prefrontal cortex could be essential because they can account for the generation of both intentional behaviors of fixed action sequences and spontaneous behaviors of pseudo-stochastic action sequences by the same mechanism.     

A neural network model for limb trajectory formation

This paper deals with the problem of representing and generating unconstrained aiming movements of a limb by means of a neural network architecture. The network produced time trajectories of a limb from a starting posture toward targets specified by sensory stimuli. Thus the network performed a sensory-motor transformation. The experimenters trained the network using a bell-shaped velocity profile on the trajectories. This type of profile is characteristic of most movements performed by biological systems. We investigated the generalization capabilities of the network as well as its internal organization. Experiments performed during learning and on the trained network showed that: (i) the task could be learned by a three-layer sequential network; (ii) the network successfully generalized in trajectory space and adjusted the velocity profiles properly; (iii) the same task could not be learned by a linear network; (iv) after learning, the internal connections became organized into inhibitory and excitatory zones and encoded the main features of the training set; (v) the model was robust to noise on the input signals; (vi) the network exhibited attractor-dynamics properties; (vii) the network was able to solve the motorequivalence problem. A key feature of this work is the fact that the neural network was coupled to a mechanical model of a limb in which muscles are represented as springs. With this representation the model solved the problem of motor redundancy.     

A Robot-Assisted Surgical System Using a Force-Image Control Method for Pedicle Screw Insertion

Objective

To introduce a robot-assisted surgical system for spinal posterior fixation that can automatically recognize the drilling state and stop potential cortical penetration with force and image information and to further evaluate the accuracy and safety of the robot for sheep vertebra pedicle screw placement.

Methods

The Robotic Spinal Surgery System (RSSS) was composed of an optical tracking system, a navigation and planning system, and a surgical robot equipped with a 6-DOF force/torque sensor. The robot used the image message and force signals to sense the different operation states and to prevent potential cortical penetration in the pedicle screw insertion operation. To evaluate the accuracy and safety of the RSSS, 32 screw insertions were conducted. Furthermore, six trajectories were deliberately planned incorrectly to explore whether the robot could recognize the different drilling states and immediately prevent cortical penetration.

Results

All 32 pedicle screws were placed in the pedicle without any broken pedicle walls. Compared with the preoperative planning, the average deviations of the entry points in the axial and sagittal views were 0.50±0.33 and 0.65±0.40 mm, and the average deviations of the angles in the axial and sagittal views were 1.9±0.82° and 1.48±1.2°. The robot successfully recognized the different drilling states and prevented potential cortical penetration. In the deliberately incorrectly planned trajectory experiments, the robot successfully prevented the cortical penetration.

Conclusion

These results verified the RSSS’s accuracy and safety, which supported its potential use for the spinal surgery.     

A neural network learns trajectory of motion from the least action principle

This paper considers the problem of training layered neural networks to generate sequences of states. Aiming at application for situations when an integral characteristic of the process is known rather than the specific sequence of states we put forward a method in which underlying general principle is used as a foundation for the learning procedure. To illustrate the ability of a network to learn a task and to generalize algorithm we consider an example where a network generates sequences of states referred to as trajectories of motion of a particle under an external field. Training is grounded on the employment of the least action principle. In the course of training at restricted sections of the path the network elaborates a recurrent rule for the trajectory generation. The rule proves to be equivalent to the correct equation of motion for the whole trajectory.     

Recurrence Network Analysis of the Synchronous EEG Time Series in Normal and Epileptic Brains

We sought to analyze the dynamic properties of brain electrical activity from healthy volunteers and epilepsy patients using recurrence networks. Phase-space trajectories of synchronous electroencephalogram signals were obtained through embedding dimension in phase-space reconstruction based on the distance set of space points. The recurrence matrix calculated from phase-space trajectories was identified with the adjacency matrix of a complex network. Then, we applied measures to characterize the complex network to this recurrence network. A detailed analysis revealed the following: (1) The recurrence networks of normal brains exhibited a sparser connectivity and smaller clustering coefficient compared with that of epileptic brains; (2) the small-world property existed in both normal and epileptic brains consistent with the previous empirical studies of structural and functional brain networks; and (3) the assortative property of the recurrence network was found by computing the assortative coefficients; their values increased from normal to epileptic brain which accurately suggested the difference of the states. These universal and non-universal characteristics of recurrence networks might help clearly understand the underlying neurodynamics of the brain and provide an efficient tool for clinical diagnosis.     

Progressive transformation of hippocampal neuronal representations in "morphed" environments

Hippocampal neural codes for different, familiar environments are thought to reflect distinct attractor states, possibly implemented in the recurrent CA3 network. A defining property of an attractor network is its ability to undergo sharp and coherent transitions between pre-established (learned) representations when the inputs to the network are changed. To determine whether hippocampal neuronal ensembles exhibit such discontinuities, we recorded in CA3 and CA1 when a familiar square recording enclosure was morphed in quantifiable steps into a familiar circular enclosure while leaving other inputs constant. We observed a gradual noncoherent progression from the initial to the final network state. In CA3, the transformation was accompanied by significant hysteresis, resulting in more similar end states than when only square and circle were presented. These observations suggest that hippocampal cell assemblies are capable of incremental plastic deformation, with incongruous information being incorporated into pre-existing representations.     

Dynamical systems for predictive control of autonomous robots

Summary Regularities in the environment are accessible to an autonomous agents as reproducible relations between actions and perceptions and can be exploited by unsupervised learning. Our approach is based on the possibility to perform and to verify predictions about perceivable consequences of actions. It is implemented as a three-layer neural network that combines predictive perception, internal-state transitions and action selection into a loop which closes via the environment. In addition to minimizing prediction errors, the goal of network adaptation comprises also an optimization of the minimization rate such that new behaviors are favored over already learned ones, which would result in a vanishing improvement of predictability. Previously learned behaviors are reactivated or continued if triggering stimuli are available and an externally or otherwise given reward overcompensates the decay of the learning rate. In the model, behavior learning and learning behavior are brought about by the same mechanism, namely the drive to continuously experience learning success. Behavior learning comprises representation and storage of learned behaviors and finally their inhibition such that a further exploration of the environment is possible. Learning behavior, in contrast, detects the frontiers of the manifold of learned behaviors and provides estimates of the learnability of behaviors leading outwards the field of expertise. The network module has been implemented in a Khepera miniature robot. We also consider hierarchical architectures consisting of several modules in one agent as well as groups of several agents, which are controlled by such networks.     

Filtering intrusion detection alarms

A Network Intrusion Detection System (NIDS) is an alarm system for networks. NIDS monitors all network actions and generates alarms when it detects suspicious or malicious attempts. A false positive alarm is generated when the NIDS misclassifies a normal action in the network as an attack. We present a data mining technique to assist network administrators to analyze and reduce false positive alarms that are produced by a NIDS. Our data mining technique is based on a Growing Hierarchical Self-Organizing Map (GHSOM) that adjusts its architecture during an unsupervised training process according to the characteristics of the input alarm data. GHSOM clusters these alarms in a way that supports network administrators in making decisions about true and false alarms. Our empirical results show that our technique is effective for real-world intrusion data.     

Using Reinforcement Learning to Provide Stable Brain-Machine Interface Control Despite Neural Input Reorganization

Brain-machine interface (BMI) systems give users direct neural control of robotic, communication, or functional electrical stimulation systems. As BMI systems begin transitioning from laboratory settings into activities of daily living, an important goal is to develop neural decoding algorithms that can be calibrated with a minimal burden on the user, provide stable control for long periods of time, and can be responsive to fluctuations in the decoder’s neural input space (e.g. neurons appearing or being lost amongst electrode recordings). These are significant challenges for static neural decoding algorithms that assume stationary input/output relationships. Here we use an actor-critic reinforcement learning architecture to provide an adaptive BMI controller that can successfully adapt to dramatic neural reorganizations, can maintain its performance over long time periods, and which does not require the user to produce specific kinetic or kinematic activities to calibrate the BMI. Two marmoset monkeys used the Reinforcement Learning BMI (RLBMI) to successfully control a robotic arm during a two-target reaching task. The RLBMI was initialized using random initial conditions, and it quickly learned to control the robot from brain states using only a binary evaluative feedback regarding whether previously chosen robot actions were good or bad. The RLBMI was able to maintain control over the system throughout sessions spanning multiple weeks. Furthermore, the RLBMI was able to quickly adapt and maintain control of the robot despite dramatic perturbations to the neural inputs, including a series of tests in which the neuron input space was deliberately halved or doubled.     

Stochastic Computations in Cortical Microcircuit Models

Experimental data from neuroscience suggest that a substantial amount of knowledge is stored in the brain in the form of probability distributions over network states and trajectories of network states. We provide a theoretical foundation for this hypothesis by showing that even very detailed models for cortical microcircuits, with data-based diverse nonlinear neurons and synapses, have a stationary distribution of network states and trajectories of network states to which they converge exponentially fast from any initial state. We demonstrate that this convergence holds in spite of the non-reversibility of the stochastic dynamics of cortical microcircuits. We further show that, in the presence of background network oscillations, separate stationary distributions emerge for different phases of the oscillation, in accordance with experimentally reported phase-specific codes. We complement these theoretical results by computer simulations that investigate resulting computation times for typical probabilistic inference tasks on these internally stored distributions, such as marginalization or marginal maximum-a-posteriori estimation. Furthermore, we show that the inherent stochastic dynamics of generic cortical microcircuits enables them to quickly generate approximate solutions to difficult constraint satisfaction problems, where stored knowledge and current inputs jointly constrain possible solutions. This provides a powerful new computing paradigm for networks of spiking neurons, that also throws new light on how networks of neurons in the brain could carry out complex computational tasks such as prediction, imagination, memory recall and problem solving.     

Application of Time Dependent Probabilistic Collision State Checkers in Highly Dynamic Environments

When computing the trajectory of an autonomous vehicle, inevitable collision states must be avoided at all costs, so no harm comes to the device or pedestrians around it. In dynamic environments, considering collisions as binary events is risky and inefficient, as the future position of moving obstacles is unknown. We introduce a time-dependent probabilistic collision state checker system, which traces a safe route with a minimum collision probability for a robot. We apply a sequential Bayesian model to calculate approximate predictions of the movement patterns of the obstacles, and define a time-dependent variation of the Dijkstra algorithm to compute statistically safe trajectories through a crowded area. We prove the efficiency of our methods through experimentation, using a self-guided robotic device.     

A Bayesian Developmental Approach to Robotic Goal-Based Imitation Learning

A fundamental challenge in robotics today is building robots that can learn new skills by observing humans and imitating human actions. We propose a new Bayesian approach to robotic learning by imitation inspired by the developmental hypothesis that children use self-experience to bootstrap the process of intention recognition and goal-based imitation. Our approach allows an autonomous agent to: (i) learn probabilistic models of actions through self-discovery and experience, (ii) utilize these learned models for inferring the goals of human actions, and (iii) perform goal-based imitation for robotic learning and human-robot collaboration. Such an approach allows a robot to leverage its increasing repertoire of learned behaviors to interpret increasingly complex human actions and use the inferred goals for imitation, even when the robot has very different actuators from humans. We demonstrate our approach using two different scenarios: (i) a simulated robot that learns human-like gaze following behavior, and (ii) a robot that learns to imitate human actions in a tabletop organization task. In both cases, the agent learns a probabilistic model of its own actions, and uses this model for goal inference and goal-based imitation. We also show that the robotic agent can use its probabilistic model to seek human assistance when it recognizes that its inferred actions are too uncertain, risky, or impossible to perform, thereby opening the door to human-robot collaboration.     

Equilibrium point control of a monkey arm simulator by a fast learning tree structured artificial neural network

A planar 17 muscle model of the monkey's arm based on realistic biomechanical measurements was simulated on a Symbolics Lisp Machine. The simulator implements the equilibrium point hypothesis for the control of arm movements. Given initial and final desired positions, it generates a minimum-jerk desired trajectory of the hand and uses the backdriving algorithm to determine an appropriate sequence of motor commands to the muscles (Flash 1987; Mussa-Ivaldi et al. 1991; Dornay 1991b). These motor commands specify a temporal sequence of stable (attractive) equilibrium positions which lead to the desired hand movement. A strong disadvantage of the simulator is that it has no memory of previous computations. Determining the desired trajectory using the minimum-jerk model is instantaneous, but the laborious backdriving algorithm is slow, and can take up to one hour for some trajectories. The complexity of the required computations makes it a poor model for biological motor control. We propose a computationally simpler and more biologically plausible method for control which achieves the benefits of the backdriving algorithm. A fast learning, tree-structured network (Sanger 1991c) was trained to remember the knowledge obtained by the backdriving algorithm. The neural network learned the nonlinear mapping from a 2-dimensional cartesian planar hand position {x, y} to a 17-dimensional motor command space {u ₁, ..., u ₁₇}. Learning 20 training trajectories, each composed of 26 sample points {{x y{,{u ₁, ..., u ₁₇} took only 20 min on a Sun-4 Spare workstation. After the learning stage, new, untrained test trajectories as well as the original trajectories of the hand were given to the neural network as input. The network calculated the required motor commands for these movements. The resulting movements were close to the desired ones for both the training and test cases.     

A Bio-Inspired Biped Water Running Robot Incorporating the Watt-I Planar Linkage Mechanism

In this paper, a biped water running robot is developed by mimicking the water-running pattern of basilisk lizards. The dynamic mechanism of the robot was studied based on Watt-I planar linkages, and the movement trajectory of the double bar Assur Group was deduced to simulate the water-running foot trajectories of the basilisk lizard. A Central Pattern Generator （CPG）-based fuzzy control method was proposed to control the robot for realizing balance control and gait adjustment. The effectiveness of the proposed control method was verified on the prototype of a water running robot （weight： 320 g）. When the biped robot is running on water, the average force generated by the propulsion mechanism is 1.3 N, and the robot body tilt angle is 5~. The experiment results show that the propulsion mechanism is effective in realizing the basilisk lizards-like water running patterns, and the CPG-based fuzzy control method is effective in keeping the balance of the robot.